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ASSESSING  ARMY  PROFESSIONAL  FORUMS  -  METRICS  FOR  EFFECTIVENESS  AND 
IMPACT 

EXECUTIVE  SUMMARY 


Research  Requirement: 

As  the  rate  of  change  in  the  operational  environment  outpaces  the  development  of 
doctrine  and  schoolhouse  instruction,  leaders  must  direct  their  own  development  in  order  to 
adaptively  and  professionally  meet  the  challenges  brought  on  by  Army  transformation.  Army 
professional  forums  (APFs),  powered  by  advances  in  collaborative  toolsets  and  multimedia 
presentation  software,  provide  a  means  for  leader  self-development  and  professional  growth. 

The  Army  senior  leadership  recognizes  the  importance  of  agility  and  professionalism  in 
its  leaders  and  is  supporting  four  active  APFs  with  nearly  twenty  more  in  development  as  part  of 
the  broader  Army  knowledge-management  initiative.  The  ability  to  assess  the  functioning  and 
impact  of  an  APF  on  individual,  unit,  and  organizational  performance  is  key  to  ensuring  that 
APFs  provide  return  on  investment  to  the  Army  and  meet  the  developmental  needs  of  their 
members.  Yet,  tools  and  methods  for  assessing  the  functioning  and  impact  of  APFs  remain 
largely  unexplored.  The  purpose  of  the  present  research  was  to  conduct  an  initial  exploration  of 
this  area. 

Procedure: 

In  this  research,  we  proposed  a  general  fi'amework  for  evaluating  APFs.  The  framework 
posits  a  causal  chain  linking  the  structure  and  characteristics  of  the  APF  ultimately  to  enhanced 
Army  effectiveness  through  activity  in  the  APF,  changes  in  individual  member  capability,  and 
improvements  in  unit  effectiveness.  We  used  the  framework  to  assess  S3-XO.net,  an  APF  geared 
towards  battalion  and  brigade  operations  officers  (S3s)  and  executive  officers  (XOs). 

Findings: 

The  proposed  APF  assessment  framework  is  an  efficient  means  for  determining  whether 
APF  structure  and  characteristics  and  APF  activity  are  adequate  for  achieving  impact.  In 
addition,  the  framework  provides  a  starting  point  for  evaluating  individual-,  unit-,  and 
organization-level  impact  through  linking  activity  in  the  forum  to  key  challenges  jointly  faced  by 
APF  members  and  the  organization. 

When  the  framework  was  used  to  assess  S3-XO.net,  we  found  that  although  the  APF  has 
a  sound  technological  foundation  it  currently  lacks  the  member  activity  necessary  to  achieve 
significant  impact.  Generating  a  higher  level  of  activity  in  the  forum  requires  the  establishment 
of  a  core  group  of  active  participants  combined  with  more  active  facilitation  of  conversation.  A 
deeper  understanding  of  the  needs  of  the  forum’s  target  audience  would  also  aid  in  effective 
facilitation  and  generation  of  activity. 
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Utilization  and  Dissemination  of  Findings: 

The  early  stages  of  an  initiative  present  the  most  valuable  opportunity  to  establish  cheeks 
that  ensure  the  initiative  is  functioning  effectively  and  meeting  organizational  goals.  Establishing 
such  checks  enhances  the  organizational  impact  of  the  initiative.  The  APF  assessment  framework 
developed  in  the  present  research  enables  the  assessment  of  APFs  during  the  early  stages  of  the 
Army  knowledge-management  initiative.  The  framework  also  serves  a  cost-effective  prerequisite 
to  comprehensive  individual,  unit,  and  organizational  effectiveness  assessment  as  the  initiative 
reaches  more  advanced  stages. 


VI 


EVALUATING  ARMY  PROFESSIONAL  FORUMS  -  METRICS  FOR  EFFECTIVENESS 
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INTRODUCTION 


It  would  be  a  profound  understatement  to  claim  that  Army  leaders  must  be  agile  in  order 
to  meet  the  demands  of  the  current  operational  environment.  At  a  time  when  the  Army’s 
organizational  structure  is  undergoing  the  most  significant  change  since  the  1960’s  (Smith, 
2005),  leaders  must  command  a  wider  range  of  missions  than  ever  before,  often  within  the 
domain  of  a  highly  adaptive,  asymmetric  threat.  Moreover,  the  increased  involvement  of  U.S. 
forces  in  full-spectrum  operations  is  changing  the  fundamental  role  of  the  Army  from 
warfighting  to  more  broadly  supporting  the  political  ends  of  the  federal  government  through 
peacekeeping,  stabilization,  counterinsurgency,  and  conventional  warfighting,  among  others.  As 
the  rate  of  change  in  the  operational  environment  outpaces  the  development  of  doctrine  and 
schoolhouse  instruction,  leaders  must  direct  their  own  development  in  order  to  adaptively  and 
professionally  meet  the  challenges  brought  on  by  Army  transformation. 

Army  Professional  forums  (APFs),  powered  by  advances  in  collaborative  toolsets  and 
multimedia  presentation  software,  provide  a  means  for  leader  self-development  and  professional 
growth.  As  members  of  an  APF,  geographically  distributed  leaders  can  directly  share  practical 
solutions,  generate  new  ideas,  and  exchange  lessons  learned  from  operational  experiences  faster 
than  the  institutional  knowledge-dissemination  process  currently  allows  (Baum,  2005).  Through 
discussion  and  the  development  of  social  networks,  APFs  also  enable  the  growth  of  a  sense  of 
professional  community  and  mutual  obligation  among  their  members,  many  of  whom  may  never 
meet  face-to-face.  The  Army  senior  leadership  recognizes  the  importance  of  agility  and 
professionalism  in  its  leaders  and  currently  supports  four  active  APFs  with  nearly  twenty  more  in 
development  as  part  of  the  broader  Army  knowledge-management  initiative  comprised  of  the 
Battle  Command  Knowledge  System  (BCKS)  and  the  Army  Knowledge  Online  (AKO)  portal. 
The  ability  to  assess  the  functioning  and  impact  of  an  APF  on  individual,  unit,  and  organizational 
performance  is  key  to  ensuring  that  APFs  provide  return  on  investment  to  the  Army  and  meet  the 
developmental  needs  of  their  members.  Until  the  present  research  was  conducted,  however,  this 
topic  was  unexplored. 

The  purpose  of  the  present  research  was  to  conduct  an  initial  effort  addressing  this  gap  in 
understanding  of  the  effectiveness  and  impact  of  APFs.  This  report  documents  the  assessment 
framework  we  designed  for  evaluating  the  functioning  and  impact  of  APFs  and  the  results  we 
achieved  when  applying  our  assessment  framework  to  an  established  APF  called  S3-XO.net. 
However,  our  goal  was  to  establish  impact-assessment  metrics  applicable  across  APFs.  This 
report  therefore  provides  guidance  for  applying  the  metrics  to  APFs  other  than  S3-XO.net.  In 
addition,  we  summarize  the  future  research  directions  necessary  to  ensure  the  generalizability  of 
the  proposed  metrics  and  to  understand  the  impact  of  knowledge  sharing  on  the  Army  more 
generally. 
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KNOWLEDGE  SHARING,  THE  ARMY  LEADER,  AND  ORGANIZATIONAL 

EXCELLENCE 


The  APE  -  Serving  the  Passionate  Professional 

Knowledge-sharing  programs  have  been  adopted  by  several  commercial  organizations 
whose  leadership  has  recognized  the  importance  of  employee  agility  and  initiative  to  competitive 
advantage  and  organizational  success.  Serving  as  the  foundation  for  best  practice  in  knowledge 
management  is  the  community  of  practice  (CoP;  Crager  &  Lemons,  2003).  Wenger,  McDermott, 
and  Snyder  (2002)  define  CoPs  as:  “groups  of  people  that  share  a  concern,  a  set  of  problems,  or  a 
passion  about  a  topic  and  who  deepen  their  knowledge  and  expertise  in  this  area  by  interacting 
on  an  ongoing  basis”  (p.  4).  Online  CoPs  provide  a  virtual  roundtable  where  people  with  similar 
concerns  can  exchange  knowledge  and  work  together  toward  innovative  practical  solutions 
(Bobrow  &  Whalen,  2002).  Online  CoPs  have  generated  significant  return  on  investment  through 
increased  productivity  and  decreased  redundancy  in  commercial  organizations  that  have 
supported  their  development  and  implemented  them  effectively  (Crager  &  Lemons,  2003). 

The  APF  represents  an  extension  of  the  online  CoP  because  it  goes  beyond  building 
practical  solutions  to  building  the  profession  itself  (Woodie,  2005).  APFs  differ  subtly  from 
online  CoPs.  Whereas  APFs  share  the  design  principles  of  online  CoPs,  the  primary  purpose  of 
APFs  is  to  support  professional  conversation  among  forum  members  through  moderated  and 
facilitated  discussion.  APF  members  do  not  share  only  a  single  common  problem  or  class  of 
problems  (e.g.,  maintenance  troubleshooting;  Bobrow  &  Whalen,  2002)  as  CoP  members  do. 
Often,  multiple  CoPs  are  situated  inside  a  single  APF.  Also,  membership  in  these  forums— unlike 
in  CoPs-consists  of  present,  future  and  past  practitioners  spanning  organizational  boundaries. 

Through  voluntary  participation,  APF  members  provide  a  positive  solution-oriented  voice 
within  the  Army  profession.  APFs  such  as  companycommand.mil  (CC.mil)  and  NCOTeam.org 
reflect  the  passion  for  the  profession  and  the  devotion  to  learning  and  personal  growth  typical  of 
U.S.  Army  leaders.  Consider  the  following  quotes  from  members  of  these  APFs: 

“I  was  on  this  site  whenever  I  logged  onto  a  computer  in  Iraq.  . . .  I'm  definitely  a 
stronger  and  more  well  rounded  NCO  [noncommissioned  officer]  because  of 
NCOTeam.org.  . . .  When  I  did  have  a  pretty  big  personal/professional  issue,  I  addressed 
my  concerns  to  my  peers  on  this  site  and  received  nothing  but  support  and  guidanee.  . . . 

An  Army  of  One  really  doesn't  mean  an  Army  of  individuals;  we  truly  are  one  team 
here.” 

-  Staff  Sergeant  JL,  Iraq 

“I  look  forward  to  . . .  see  what  great  information  you  all  are  going  to  put  out.  I  can’t  wait 
to  share  it  with  the  other  platoon  leaders  and  the  Company  XO  [executive  officer].” 

-  Paul 

“I  really  appreciate  this  web-site,  and  the  newsletter.  I  arrived  in  Iraq  in  April  and  have 
done  a  lot  of  growing  up.  CC[.mi[]  was  a  great  help  through  some  tough  experiences. 

Thanks.”  -  Captain  K,  from  IZ 
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“This  resource  has  been,  and  continues  to  be,  an  invaluable  tool  for  professionals  to 
interact  and  benefit  from  the  collective  knowledge  of  others  in  our  profession  of  arms... 

This  forum  is  simply  a  tool  that  every  NCO,  from  infancy  to  seasoned  veteran,  can  be 
assured  that  they  are  following  in  the  footsteps  of  excellence. . .” 

-  Sergeant  JMD,  Assistant  Commandant,  U.S.  Army  NCO  Academy 

CC.mil  and  NCOTeam.org  have  been  heralded  far  and  wide  in  the  media,  featured  in  best 
practice  reports  (Snyder  &  Briggs,  2003),  books  (Dixon,  Allen,  Burgess,  Kilner,  &  Schweitzer, 
2005),  news  articles  [Baum,  2005;  13*  Corps  Support  Command,  2003],  and  even  discussed  on 
the  radio  (National  Public  Radio,  January  25,  2005).  They  are  widely  recognized  for  the  strong 
interpersonal  bonding  and  active  knowledge  sharing  among  their  members.  Long-standing  and 
well-established  APFs  such  as  these  represent  the  gold  standard  when  evaluating  the 
effectiveness  of  efforts  to  cultivate  and  sustain  APF  activity. 

Turning  Passion  Into  Impact 

Although  it  may  seem  common  sense  that  active  APFs  have  an  impact  on  organizational 
performance,  an  understanding  of  how  APF  activity  generates  organizational  effectiveness  is  still 
in  its  infancy.  Put  another  way,  it  remains  to  be  explained  how  passionate  activity  in  an  APF 
results  in  individual,  interpersonal,  team,  community,  and  unit  growth  and  how  this  growth 
ultimately  promotes  organizational  excellence.  Such  an  understanding  is  necessary  to 
demonstrate  organizational  impact,  and  is  required  to  assure  high-level  decision  makers  that 
money  invested  in  staffing  and  running  APFs  is  money  not  better  invested  in  other  leader- 
development  endeavors.  In  addition,  understanding  the  process  by  which  APFs  generate 
organizational  excellence  helps  knowledge-management  administrators  and  designers  determine 
how  best  to  facilitate  the  effectiveness  of  APFs  by  addressing  weaknesses  and  enhancing 
strengths.  Understanding  the  impact  process  requires  that  a  causal  Ifamework  be  developed  to 
link  activity  in  an  APF  ultimately  to  organizational  excellence  (Crager  &  Lemons,  2003;  Lesser 
&  Storck,  2001;  Snyder  &  Briggs,  2003).  Case  studies  Ifom  commercial  organizations  provide 
the  outline  of  such  a  causal  framework  and  provide  the  first  evidence  that  efforts  to  support 
problem-focused  knowledge  sharing  (i.e.,  in  CoPs)  can  have  a  tremendous  return  on  investment 
(e.g.,  Crager  &  Lemons,  2003). 

Developing  a  causal  framework  for  understanding  Army  effectiveness  is  an  exceedingly 
difficult  task  for  multiple  reasons.  First,  Army  effectiveness  is  not  easily  quantified  (i.e.,  it  is  not 
reflected  in  units  of  production  or  revenue  dollars)  or  even  conceptualized  (e.g.,  the  Army  is 
successful  both  when  it  wins  conflicts  and  when  it  prevents  conflicts  through  stabilization  and 
reconstruction).  Second,  when  attempting  to  determine  the  impact  of  a  particular  APF,  there  is 
the  problem  that  nearly  every  aspect  of  Army  effectiveness  is  determined  by  the  collective 
activity  of  members  of  that  APF  and  other  individuals.  In  business,  by  contrast,  members  of  a 
CoP  are  more  likely  to  be  focused  on  a  circumscribed  problem  set  and  to  represent  a  department 
that  makes  a  discrete  contribution  to  productivity  (e.g.,  repair  service,  see  Bobrow  &  Whalen, 
2002).  Therefore,  the  impact  of  knowledge  sharing  on  departmental  effectiveness  can  be  more 
easily  identified  and  departmental  impact  on  overall  organizational  performance  can  be 
estimated.  Third,  through  BCKS  and  AKO,  leaders  have  a  very  broad  range  of  resources 
available  to  them  for  knowledge  acquisition  and  sharing.  The  adaptive  cross-membership  of 
individual  leaders  in  multiple  communities  makes  isolating  the  impact  of  a  particular  APF  on 
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organizational  effectiveness  difficult.  Finally,  adaptive  cross-membership  in  multiple 
communities  (including  those  available  on  secure  Internet  connections  and  those  on  insecure 
government  and  public  connections)  has  implications  for  where  leaders  will  go  to  meet  their 
needs  given  that  they  have  limited  time  to  spend.  Understanding  how  activity  in  an  APF  affects 
unit  and  organizational  effectiveness  therefore  requires  understanding  how  the  environment  in 
which  leaders  are  operating  (e.g.,  wartime  vs.  peacetime,  known  vs.  unknown  enemy,  and  so  on) 
influences  the  needs  of  forum  members  and  the  ability  to  meet  these  needs  in  the  forum. 

Demonstrating  Impact  to  High-Level  Decision  Makers 

The  difficulty  of  developing  a  causal  framework  does  not  mean  that  an  attempt  should 
not  be  made  to  understand  how  APF  activity  affects  organizational  performance.  If  the  present 
analysis  stopped  at  simply  describing  APF  activity  (though  this  is  an  important  aspect  of 
determining  impact),  then  the  purpose  of  our  research  would  not  be  fulfilled.  High-level  decision 
makers  would  not  have  the  information  they  require,  and  a  bridge  would  not  be  built  between 
APF  enthusiasts  and  those  who  must  make  strategic  decisions  about  how  to  allocate  limited 
resources.  Moreover,  valuable  information  that  could  be  used  to  enhance  the  effectiveness  of 
current  APFs  or  guide  the  development  of  future  APFs  would  not  be  gathered.  What  the 
challenge  in  both  developing  a  meaningful  causal  framework  and  demonstrating  impact  means  is 
that  impact  must  be  clearly  understood  within  the  context  of  the  APF  of  interest  and  its  members. 
There  is  useful  guidance  for  developing  such  an  understanding  in  best-practice  reports  such  as 
Crager  and  Lemons  (2003)  and  Lesser  and  Storck  (2001).  The  key  is  to  align  metrics  of  APF 
activity  with  organizational  objectives  and  mission  using  as  a  shared  reference  point  the  tasks  the 
APF  members  must  perform  to  do  their  jobs  effectively.  A  rational  analysis  of  this  alignment  is  a 
widely  recognized  critical  first  step  not  only  in  evaluating  the  impact  of  knowledge  management 
[see,  e.g.,  Hanley,  2003;  Crager  &  Lemons,  2003;  Lesser  &  Storck,  2001 ;  Department  of  the 
U.S.  Navy,  2001]  but  also  for  designing  strategies  that  result  in  the  successful  creation, 
sustainment,  and  growth  of  a  knowledge  forum  itself  (Snyder  &  Briggs,  2003). 

A  Proposed  Causal  Framework  Linking  APF  Activity  to  Organizational  Effectiveness 

Figure  1  shows  our  proposed  causal  framework  for  linking  APF  activity  to  organizational 
effectiveness.  Each  component  of  the  framework  is  explained  in  the  following  five  sections.  The 
assessment  criteria  and  metrics  following  the  presentation  of  the  framework  were  determined 
through  a  combination  of  literature  review  and  user  and  stakeholder  interviews  (see  Appendix 
A). 


APF  Structure  and  Characteristics 

The  structure  and  characteristics  of  the  APF  serve  as  the  foundation  of  our  proposed 
causal  framework.  Structure  and  characteristics  represent  the  features  of  the  APF  that  support 
knowledge  retrieval  and  the  development  of  social  connections  that  lead  to  discussion  and  the 
development  of  new  knowledge.  These  features  are  the  determinants  of  APF  activity  and  impact. 
APF  structure  and  characteristics  may  include  interface  design  features/usability  (e.g., 
organization  of  knowledge)  and  community  features  (e.g.,  member  bios),  as  well  as  the 
administrative  backbone  of  the  APF,  including  the  composition  of  the  core  support  team,  the 
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organizational  support  for  the  APF,  and  the  alignment  of  the  APF  goals  with  the  organizational 
mission,  among  other  considerations. 


SPF  Structure  and  Characteristics 
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Figure  1 .  Proposed  Framework  for  Linking  APF  Activity  to  Organizational  Effectiveness 


Social  and  Intellectual  Capital 

Enabled  by  APF  structure  and  characteristics  is  the  development  of  social  and  intellectual 
capital.  If  the  APF  is  designed  and  administered  effectively,  members  will  be  able  to  retrieve 
actionable  (i.e.,  relevant  and  timely)  knowledge  from  the  community.  Effective  APF  design  and 
facilitation  will  also  enable  members  to  participate  actively  in  developing  a  network  of 
individuals  with  whom  to  discuss  novel  problems  and  collaboratively  generate  new  knowledge. 
Discussion  and  the  development  of  soeial  capital  are  key  characteristics  of  APFs  that 
differentiate  them  from  knowledge  repositories  or  portals  (Kendall  &  McHale,  2003;  Schweitzer, 
2003).  Forum  members  actively  develop  social  and  intellectual  capital  for  there  to  be  a 
demonstrable  impact  of  the  APF  on  individual,  unit,  and  organizational  performance. 
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Leader  Competence  and  Professionalism 


The  development  of  social  and  intellectual  capital  leads  to  improved  leader  competence 
and  sense  of  professionalism.  Specifically,  the  acquisition  and  development  of  actionable 
knowledge  enhances  individual  technical,  tactical,  and  conceptual  competence.  Leaders  are 
better  equipped  to  handle  such  challenges  as  applying  the  military  decision  making  process 
(MDMP)  or  leveraging  technology,  for  example,  beeause  they  have  been  exposed  to  the  effective 
methods  their  peers  have  used  to  develop  planning  standard  operating  proeedures  or  to  work  with 
emerging  digital  systems. 

Professional  self-identity  also  develops  through  active  knowledge  sharing  and 
development  because  it  is  through  this  activity  that  the  leader  actively  participates  in  gaining  a 
concrete  sense  of  his  role  in  his  current  duty  position  and  the  associated  tasks,  duties,  and 
responsibilities  he  must  perform.  Participating  in  the  APF  also  fosters  the  skills  and  attitudes 
necessary  for  continuous  professional  self-development. 

Discussion  with  peers  promotes  a  sense  of  professional  community,  a  sense  that  one 
belongs  to  a  group  of  people  who  share  similar  duties  and  responsibilities,  who  play  a  similar 
role  in  the  larger  organization,  who  face  similar  problems  and  challenges,  and  to  whom  one  is 
responsible  for  being  respectful  and  helpful. 

Interpersonal  competency  is  enhanced  through  storytelling  in  which  leaders  share  how 
they  handled  novel  and  sensitive  interpersonal  issues.  Leadership  effectiveness  also  is  enhanced 
through  storytelling  in  which  leaders  share  how  they  handled  authentic  leadership  issues.  It  is 
through  this  storytelling  that  the  tacit  knowledge  critical  to  leadership  effectiveness  is  acquired 
(Cianciolo,  Antonakis,  &  Sternberg,  2004). 

Organizational  Effectiveness  -  Unit 

Organizational  effectiveness  at  the  unit  level  is  a  result  of  the  development  of  leader 
competence  and  professionalism.  The  expert  and  mature  leader  is  more  technically  competent,  as 
well  as  a  more  effective  role  model  and  team  player,  which  is  critical  in  supporting  the  day-to- 
day  operations,  morale,  professionalism,  and  combat  effectiveness  of  the  unit. 

Organizational  Effectiveness  -  The  Army 

Organizational  effectiveness  at  the  Army  level  results  in  large  part  from  the  enhanced 
functioning  of  its  tactical  warfighting  units.  In  short,  these  units  must  see  first,  understand  first, 
act  first,  and  finish  decisively  in  full-spectrum  operations.  The  organizational  effectiveness  of  the 
Army  also  is  enhanced  by  achieving  better  results  with  fewer  resources  and  retaining  more  high- 
quality  personnel.  This  result  is  achieved,  in  part,  through  increased  professional  self¬ 
development  on  the  part  of  Army  personnel  and  increased  satisfaction  with  Army  life. 

Each  of  these  sections  of  the  causal  framework,  together  with  implications  for  metrics 
and  their  associated  measures  will  be  discussed  in  detail  in  the  following  sections. 
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ASSESSING  APF  STRUCTURE  AND  CHARACTERISTICS 


As  described  previously,  an  APF’s  structure  and  characteristics  determine  the  level  of 
activity  in  the  APF  and  the  impact  it  can  be  expected  to  have  on  performance.  If  the  APF  does 
not  have  a  solid  foundation,  the  forum  will  not  be  active  and  cannot  be  expected  to  generate 
impact.  Similarly,  if  an  APF  is  found  to  be  inactive  and  incapable  of  generating  impact,  the  root 
of  the  problem  may  be  revealed  by  evaluating  the  effectiveness  of  the  APF’s  structure  and 
characteristics.  In  order  to  determine  the  effectiveness  of  APF  structure  and  characteristics,  the 
following  three  broad  aspects  of  the  APF  foundation  should  be  assessed  in  detail:  (1)  the  plan  on 
which  the  development  of  the  forum  is  based;  (2)  the  personnel  structure  that  supports  forum 
growth  and  activity;  and  (3)  the  degree  to  which  the  forum  meets  the  general  functional 
requirements  for  online  knowledge  sharing  and  community  building.  An  explanation  of  each  of 
these  aspects  of  APF  structure  and  characteristics  is  presented  below,  followed  by  proposed 
metrics  for  assessing  these  aspects. 

Structure  and  Characteristics  Necessary  for  Facilitating  APF  Activity 

The  Plan 

Several  sources  converge  on  the  importance  of  starting  the  APF  development  process 
with  an  initial  plan  for  the  forum  (e.g.,  BCKS,  2005;  Dixon  et  al.,  2005;  Gerber,  2003;  Glennie  & 
Hickok,  2003;  DON,  2001;  Wenger  et  al.,  2002).  The  plan  ensures  that  the  design  of  the  forum  is 
aligned  with  member  needs,  which  stimulates  activity,  and  with  the  mission  and  goals  of  the 
parent  organization,  which  translates  activity  into  impact.  The  key  features  of  the  plan  include: 

A  statement  of  the  purpose  of  the  forum 

The  envisioned  impact  of  the  forum  and  the  criteria  to  be  used  to  assess  this  impact 
'T  A  description  of  the  target  audience,  including  the  needs  of  this  audience 
'T  A  statement  of  the  cultural  norms,  core  values,  or  conventions  to  be  honored 
'T  A  specification  of  the  roles  and  responsibilities  of  the  supporting  personnel 

A  functional  and  technical  specification  of  the  online  meeting  space 

According  to  Dixon  et  al.  (2005),  defining  a  forum’s  purpose  provides  a  framework  for 
the  administrative  team  to  use  in  determining  the  scope  of  their  efforts  —  to  determine 
membership  criteria,  the  nature  of  content  and  its  organization,  and  the  functional  requirements 
of  the  forum.  The  forum  type,  or  general  purpose,  may  be  (a)  helping,  where  the  purpose  is  to 
provide  a  place  where  members  help  each  other  solve  practical  problems;  (b)  best  practice, 
where  the  purpose  is  to  develop  and  disseminate  best  practice  information;  (c)  knowledge 
stewarding,  where  the  purpose  is  to  develop  tools,  insights,  and  approaches  needed  by  forum 
members  in  the  field;  or  (d)  innovating,  where  the  purpose  is  to  develop  highly  innovative 
solutions  and  ideas  (Wenger  et  al.,  2002).  Wenger  et  al.  recommend  that  a  forum  be  focused  on 
one  of  these  general  purposes  in  order  to  avoid  overwhelming  forum  members.  In  addition,  the 
forum’s  general  purpose  should  be  determined  by  the  overall  strategic  intent  of  developing  the 
forum  in  the  first  place. 
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The  envisioned  impact  of  the  forum  directly  ties  the  forum’s  purpose  to  achieving  the 
parent  organization’s  mission  and  goals.  The  statement  of  envisioned  impact  should  specify  how, 
if  the  forum  evolves  according  to  its  purpose,  it  will  influence  the  attitudes  and  behavior  of  its 
members  in  a  way  that  benefits  the  organization  (e.g.,  fosters  organizational  commitment, 
shortens  learning  time,  etc.,  see  Lesser  &  Storck,  2001).  This  specification  should  result  in 
specific,  concrete  criteria  to  be  used  for  later  evaluating  the  impact  of  the  forum. 

Description  of  the  target  audience  and  its  needs  is  critical.  Wenger  et  al.  (2002) 
recommend  that  the  scope  of  the  target  audience  be  wide  enough  to  encompass  new  people  and 
ideas,  but  not  so  wide  that  members  frequently  are  not  interested  in  the  content  posted  or  find  it 
difficult  to  retrieve  the  information  they  seek  (see  also  Gerber,  2003;  Stuckey  &  Smith,  2004). 
Ultimately,  the  target  audience  should  include  people  who  know  and  people  who  need  to  know 
(Dixon  et  al.,  2005).  An  important  aspect  of  analyzing  the  needs  of  the  target  audience  is 
determining  how  essential  the  planned  forum  is  to  meeting  these  needs  (BCKS,  2005).  It  should 
be  determined  that  the  needs  of  the  target  audience  (a)  can  actually  be  met  by  the  planned  forum; 
and  (b)  cannot  be  better  met  by  some  other  means.  In  addition,  once  the  target  audience’s  needs 
are  identified,  those  needs  that  are  particularly  important  to  the  organization  and  those  needs 
about  which  forum  members  are  especially  passionate  should  be  selected  for  initial  focus  in  the 
forum. 


The  plan  should  include  a  statement  of  the  cultural  norms,  core  values,  or  conventions  to 
be  honored  because  such  a  statement  sets  the  tone  for  the  forum.  Setting  the  correct  tone  for  the 
forum  will  build  members’  trust  that  knowledge  shared  or  generated  in  the  forum  will  benefit 
their  development  and  that  participating  in  the  forum  will  not  have  negative  implications  for  their 
career.  Such  trust  is  critical  to  the  health  of  the  forum  (Gerber,  2003).  Moreover,  trust  in  the 
forum,  built  through  shared  values,  will  help  keep  members  engaged  in  the  forum  (Prevou  & 
Burgess,  2003). 

Another  key  component  of  the  plan  is  the  specification  of  the  roles  and  responsibilities  of 
the  supporting  personnel.  Numerous  sources  converge  on  the  following  general  categories  of 
supporting  personnel:  leader,  core  group,  facilitators,  and  technical  support  (Dixon  et  al.,  2005; 
DON,  2001 ;  Snyder  &  Briggs,  2003;  Wenger  et  al.,  2002).  The  plan  should  designate  the  speeific 
tasks  assigned  to  each  category  of  personnel,  who  will  perform  these  tasks,  and  how  mueh  time 
personnel  are  expected  to  spend  on  these  tasks  relative  to  other  job  duties.  The  communication 
channels  between  categories  of  personnel  should  also  be  identified  in  order  to  facilitate  timely 
and  effective  communication. 

The  final  component  of  the  forum  plan  is  the  functional  and  technical  specification  of  the 
online  meeting  space.  This  specification  should  include  a  description  of  what  activities  the  online 
meeting  space  will  support  (e.g.,  discussion,  solution  sharing,  social  networking,  etc.),  the 
interface  features  or  functionalities  the  online  meeting  space  must  have  to  support  these  activities 
(search  capability,  conversation  posting  and  archiving,  etc.),  and  the  hardware/software  and 
security  requirements  necessary  to  successfully  implement  the  online  meeting  space.  Because  the 
same  software  platform  (Tomoye  Simplify)  will  be  used  to  support  all  APFs,  this  component  of 
the  forum  plan  is  of  greater  importance  for  identifying  future  functional  and  technological 
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requirements  for  APFs  as  a  whole  than  for  identifying  the  same  requirements  for  an  individual 
forum. 


Personnel 

As  described  above,  there  are  four  general  categories  of  personnel  required  to  initiate  and 
sustain  a  healthy  APF.  These  categories  and  their  requirements  for  serving  the  healthy  APF  are 
described  in  more  detail  below.' 

The  leader,  or  community  coordinator,  of  the  forum  serves  the  critical  leadership 
functions  of  providing  an  overall  vision  for  the  forum  and  helping  supporting  personnel  to 
achieve  this  vision  through  a  well  developed  professional  network  and  his/her  role  within  the 
formal  organization  (Wenger  et  al.,  2002).  The  leader  must  be  knowledgeable  about  the  domain 
(i.e.,  the  general  topic  area  of  interest  to  forum  members)  addressed  in  the  forum,  though  not 
necessarily  a  subject  matter  expert,  and  must  have  a  passion  for  the  professional  development  of 
the  forum  members  (Wenger  et  al.,  2002).  Forum  leaders  must  also  take  the  initiative  in  fostering 
the  forum’s  long-term  health  even  in  the  absence  of  subject  matter  expertise  (Wenger  et  al., 
2002).  Most  importantly,  the  leader  must  be  supported  and  willing  to  commit  between  20  and  50 
percent  of  his/her  time  to  leading  the  forum.  A  common  pitfall  in  forum  leadership  is  to  spend 
more  time  cultivating  activity  in  the  online  space  of  the  forum  (a  role  of  the  core  group  and 
facilitators,  see  below)  than  cultivating  offline  enthusiasm  for  the  forum  through  active 
marketing  and  networking  with  community  members  (Wenger  et  al.,  2002). 

The  core  group  is  the  foundation  of  the  APF.  Building  a  core  group  is  among  the  critical 
first  steps  in  establishing  an  active  community  because  it  is  through  this  group  of  highly  active 
members  that  the  forum  establishes  its  identity  and  sets  the  conditions  for  sustained  growth 
(Dixon  et  al.,  2005;  Wenger  et  al.,  2002).  The  core  group  should  comprise  knowledgeable  and 
experienced  members  of  the  forum  target  audience  because  they  must  use  their  subject  matter 
expertise  to  make  judgments  about  the  importance  and  relevance  of  knowledge  in  the  forum,  and 
to  analyze  and  synthesize  the  knowledge  (Dixon  et  al.,  2005;  Department  of  the  U.S.  Navy, 
2001).  In  addition,  the  members  of  the  core  group  must  have  a  strong,  wide  social  network  in 
order  to  have  thorough  visibility  on  the  needs  and  interests  of  the  target  audience,  to  connect 
members  of  the  target  audience  to  willing  experts  who  can  help,  to  gamer  support  for  the  forum 
in  its  initial  stages  of  growth,  and  to  be  an  advocate  for  the  forum  during  institutionalization 
(Dixon  et  al.,  2005).  Moreover,  because  the  core  group  is  the  heart  of  forum  facilitation,  its 
members  should  have  subject  matter  expertise  in  online  professional  forums  (both  the  technical 
and  behavioral  aspects)  or  at  least  facilitating  collaborative  activity  online.  Finally,  and  perhaps 
most  importantly,  the  core  group  must  have  a  passion  for  developing  the  profession,  particularly 
as  it  relates  to  the  professional  growth  of  the  forum’s  target  audience.  This  passion  is  critical  to 
attracting  new  members  to  the  site,  as  well  as  expert  contributors  and  financial  supporters,  and  to 
maintaining  the  level  of  involvement  necessary  to  sustain  the  forum  (Stuckey  &  Smith,  2004). 


'  It  should  be  noted  that  with  the  possible  exception  of  forum  leaders,  individuals’  duties  may  fall  under  multiple 
categories  such  that,  for  example,  a  member  of  the  core  group  also  facilitates  conversation  or  a  facilitator  provides 
technical  support.  It  is  more  important  that  the  functions  of  the  four  personnel  categories  be  served,  than  that  each 
category  is  comprised  of  different  individuals. 
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Because  professional  forums  do  not  maintain  themselves,  the  activity  of  the  facilitators  is 
absolutely  critical  to  the  health  of  the  forum  (Stuckey  &  Smith,  2004).  Facilitators  play  the  key 
roles  of  setting  the  tone  in  the  forum  and  responding  to  member  needs  for  knowledge  and 
discussion  in  particular  topic  areas.  To  set  the  tone,  facilitators  must  monitor  discussion  threads 
to  keep  conversation  moving  in  a  productive  direction  and  to  ensure  that  the  core  values  of  the 
forum  are  being  honored.  Facilitators  respond  to  member  needs  in  a  variety  of  ways,  including 
(a)  seeking  out  and  organizing  relevant  knowledge  that  can  be  posted  in  the  forum  for  easy 
viewing,  retrieval  and  discussion;  (b)  identifying  subject  matter  experts  who  can  contribute  to  the 
discussion  of  particular  topics;  and  (c)  linking  members  with  questions  to  members  with  answers 
(Gerber,  2003;  Stuckey  &  Smith,  2004). 

In  the  initial  stage  of  forum  development,  the  leader  and  core  group  work  together  to 
identify  and  send  out  personal  invitations  to  representatives  of  the  target  audience  (“pointmen”) 
to  participate  in  the  forum  and  spread  the  word  that  the  forum  is  up  and  running  (Dixon  et  al., 
2005).  These  individuals  also  identify  “command  contacts,”  subject  matter  experts  who  are 
willing  to  support  the  professional  development  of  the  forum  members  (Dixon  et  al.,  2005). 
Facilitators  also  participate  in  identifying  and  contacting  pointmen  and  command  contacts, 
especially  as  the  forum  begins  to  grow.  The  leader,  core  group,  and  facilitators  must  form  a 
cohesive,  mutually  supportive,  and  passionate  team  through  open  communication,  camaraderie, 
and  desire  to  grow  the  profession  (Dixon  et  al.,  2005). 

The  technical  support  staff  translates  functional  requirements  into  technical  requirements 
and  interface  design  features.  The  technical  support  staff  is  therefore  responsible  for  designing 
and  implementing  the  collaborative  environment,  to  include  software,  hardware,  Internet,  and 
security  requirements.  Moreover,  technical  support  staff  members  must  maintain  the 
collaborative  environment,  ensuring  that  (a)  knowledge  and  individuals  within  the  forum  are 
easy  to  locate;  (b)  interface  features  (e.g.,  hotlinks,  forms)  work  as  intended;  (c)  additional 
features  needed  by  forum  members  or  support  personnel  (e.g.,  content  rating  forms,  survey 
forms,  synchronous  chat)  are  implemented;  (d)  database  functions  (e.g.,  address  book,  activity 
metrics,  knowledge  search)  work  properly;  and  (e)  facilitators  and  other  support  personnel  are 
trained  to  use  the  software.  Technical  support  staff  members  should  be  experts  in  both 
knowledge  management  and  information  technology. 

Functional  Requirements 

Functional  requirements  are  those  requirements  the  online  meeting  space  must  meet  in 
order  to  provide  an  environment  suitable  for  an  online  community.  Fundamentally,  this 
collaborative  environment  must  support  APF  members’  ability  to  share  and  discuss  content  and 
to  develop  social  connections  (Nahapiet  &  Ghoshal,  1998;  Stuckey  &  Smith,  2004).  The  online 
meeting  space  must  therefore  provide  a  place  where  APF  members  can  engage  in  discussion, 
especially  asynchronous  discussion  (see  Schweitzer,  2003),  about  the  topics  that  interest  them. 
Members  must  be  able  to  locate  content  of  interest  and  identify  other  members  with  whom  it  will 
be  beneficial  to  discuss  that  content.  Because  the  typical  APF  user  is  likely  to  be  limited  in  his 
free  time,  he  must  be  able  to  locate  content  and  other  members  extremely  quickly  (i.e.,  “three 
clicks”  or  less),  else  he  will  find  some  other  means  to  meet  his  needs  or  simply  not  meet  them  at 
all. 


10 


The  first  step  in  meeting  these  functional  requirements  is  that  the  APF  itself  be  easy  to 
locate  and  easy  to  join.  This  requires  that  prospective  members  either  be  able  to  locate  the  APF 
through  BCKS,  AKO,  The  Center  for  Army  Lessons  Learned  (CALL),  Google,  installation 
homepages,  or  some  other  readily  available  means  or  that  they  have  heard  of  the  APF  through  its 
members,  the  APF  leadership  and/or  core  group,  instructors  in  the  schoolhouse,  commanding 
officers,  mentors,  or  other  colleagues  (e.g.,  observer/controllers  at  the  combat  training  centers). 
Prospective  members  must  be  able  to  join  the  forum  with  minimal  effort  (e.g.,  application  forms 
are  simple  to  fill  out)  and  there  must  be  a  very  short  time  lag  between  applying  for  membership 
and  being  granted  membership  (Prevou  &  Burgess,  2003).  Importantly,  potential  APF  members 
must  feel  that  membership  restrictions  are  logical  and  fair  given  the  mission  of  the  forum,  else 
they  will  feel  overly  managed  from  the  top  down  and  buy-in  to  the  forum  will  be  diminished. 
Members  should  also  be  personally  welcomed  to  the  forum  when  they  join  in  order  to  set  the 
tone  that  the  community  is  there  for  them  and  recognizes  the  importance  of  their  membership 
(Prevou  &  Burgess,  2003).  Additional  welcoming  gestures  that  may  set  the  tone  and  foster  a 
sense  of  community  include  a  frequently  asked  questions  list,  a  hands-free  tutorial  of  the  online 
meeting  space,  an  APF  mission  statement  and  history,  and  other  background  information  (Prevou 
&  Burgess,  2003).  In  addition,  a  statement  of  the  core  values  and  conventions  may  be  useful,  as 
well  as  an  area  for  user  comments  that  fosters  a  sense  of  belonging  to  the  online  community. 

A  critical  functionality  of  the  online  meeting  space  is  the  support  of  vibrant  conversation 
among  forum  members.  It  is  this  functionality  that  distinguishes  the  discussion  forum  from  a 
knowledge  repository,  a  distinction  that  determines  the  generation  of  new  knowledge  and  the 
growth  of  the  community.  Asynchronous  discussion  is  the  backbone  of  successful  discussion 
forums,  with  synchronous  discussion  less  frequently  found  (Schweitzer,  2003).  The  online 
meeting  space  must  therefore  enable  the  storage  and  meaningful  organization  of  conversation 
threads  and  allow  forum  members  to  easily  view  conversations  and  post  their  own  contributions. 
Features  of  the  online  meeting  space  should  also  help  members  evaluate  the  value  of  a 
conversation  through  indicating  (a)  the  general  topic  of  the  conversation;  (b)  the  number  of  posts 
involved  in  the  conversation;  (c)  the  date  of  the  initial  and  most  recent  conversation  posts;  (d)  the 
contributors  of  each  post  in  the  conversation;  and  (e)  member  ratings  of  the  value  of  the 
conversation.  It  must  be  remembered,  however,  that  vibrant  conversation  ultimately  springs  from 
a  dedicated  core  group  and  facilitators  who  focus,  protect,  and  foster  the  conversation  (see  above 
section,  also  Dixon  et  al.,  2005;  Stuckey  &  Smith,  2004),  no  matter  how  easy  to  use  the 
discussion  functionality  of  the  online  meeting  space  is. 

Once  in  the  online  meeting  space,  quickly  locating  and  retrieving  content  (including 
discussion  threads)  requires  that  it  be  organized  into  topic  and  sub-topic  areas  of  greatest 
relevance  to  forum  members,  and  that  the  most  important  and  recent  topics  be  placed 
prominently  on  the  very  front  page  of  the  forum  (Dixon  et  al.,  2005).  In  addition,  there  must  be  a 
robust  search  capability  that  supports  searching  the  entire  forum  or  selected  communities  in  the 
forum  as  well  as  searching  for  content  using  a  variety  of  user  inputs  (e.g.,  title,  author,  key 
words,  content  type,  and  so  on).  Awareness  of  content  on  the  site  can  also  be  accomplished 
through  “featured  content”  and  “recent  discussions”  areas  within  each  topic,  “flags”  indicating 
which  topic  and  sub-topic  has  new  content,  direct  links  to  content  outside  the  forum,  links 
between  related  content  inside  the  forum,  and  regular  newsletters  featuring  additions  to  the 
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forum.  Members  should  be  able  to  identify  and  indicate  which  topics  are  the  most  popular  and 
which  contributions  are  the  most  useful  (e.g.  through  metrics  to  indicate  number  of  posts  to  a 
particular  topic  and  by  using  rating  functions). 

Finding  people  is  as  important  to  the  health  of  the  APF  as  finding  content.  Forum 
members  must  be  able  to  identify  other  members,  their  background,  and  how  to  contact  them. 
Member  bios  must  include  information  that  will  help  members  to  determine  whether  a  particular 
person  should  be  targeted  for  discussion  or  to  establish  the  credibility  of  a  person  who  has  posted 
content.  This  information  should  include  branch,  current  duty  position,  assignment  history, 
combat  experience,  and  personal  areas  of  expertise,  among  others.  Each  member  bio  should 
include  a  list  of  how  many  posts  the  individual  has  made  with  links  to  each  of  these  posts.  As 
with  content,  there  should  be  a  robust  search  capability  for  locating  people.  The  expertise  of 
particular  forum  members  may  be  highlighted  through  a  “featured  people”  area  or  a  regular 
newsletter.  To  facilitate  a  sense  of  community,  an  area  indicating  who  is  currently  logged  in  to 
the  forum  is  claimed  to  be  beneficial  (Dixon  et  ah,  2005).  Such  a  feature  allows  members  to 
characterize  the  type  of  members  who  typically  visit  the  forum.  In  an  APF  that  supports 
synchronous  chat,  such  a  feature  allows  members  to  determine  whether  there  is  an  expert 
currently  online  with  whom  a  chat  might  be  advantageous. 

Technical  Requirements 

The  speed,  accessibility,  and  security  of  the  network  in  which  the  online  meeting  space  is 
situated  are  clearly  important  considerations  for  the  leadership  and  administrative  support  of  the 
professional  forum  to  consider.  The  cost,  flexibility,  and  organizational  fit  of  the  software  used 
to  generate  the  online  meeting  space  are  additional  important  factors.  Decisions  about 
networking  and  software  that  take  these  factors  into  consideration  serve  organizational  needs 
(e.g.,  cost  savings,  political  posturing,  and  information  security)  as  much  or  more  than  the  needs 
of  the  forum  members,  and  for  this  reason  are  considered  beyond  the  scope  of  the  present 
analysis. 


Structure  and  Characteristics  -  A  Proposed  Framework  for  Assessment 

To  assess  APF  structure  and  characteristics,  a  red-amber-green  rating  system  can  be  used 
in  order  to  translate  qualitative  information  into  “analytic”  metrics  (i.e.,  metrics  that  will  allow 
rough  comparisons  between  aspects  of  structure  and  characteristics  within  or  across  APFs).  This 
system,  which  allows  a  combined  red/amber  and  amber/green  rating,  is  essentially  an  alternative 
format  to  a  five-point  rating  scale.  The  red- amber-green  system  is  preferable  to  a  numerical 
rating  scale  so  that  (a)  status  ratings  will  communicate  the  predicted  impact  of  the  rated  element 
on  the  health  of  the  forum;  and  (b)  the  use  of  numbers  does  not  mislead  the  reader  to  think  that 
the  rating  system  has  quantitative  properties  (e.g.,  an  interval  or  ratio  scale)  that  it  does  not.  The 
assignment  of  a  status  rating  to  any  element  is  a  subjective  decision,  based  on  an  understanding 
of  best  practice  in  knowledge  sharing,  and  ratings  can,  at  best,  be  thought  to  fall  on  an  ordinal 
scale.  The  assessment  framework  presented  in  this  section  features  the  structure  and 
characteristics  elements  for  which  status  should  be  assessed,  and  a  description  of  the  rating 
criteria  (“red,”  “amber,”  and  “green”)  for  each  of  these  elements.  Overall  Status  can  be 
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determined  by  taking  the  average  of  the  component  status  ratings  featured  in  each  table  if  the 
red-amber-green  scale  were  treated  like  its  analog,  the  1-5  ordinal  rating  seale. 
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Table  1.  Assessment  Framework  for  the  APF  Plan 


Table  3.  Assessment  Framework  for  Functional  Requirements 


ASSESSING  SOCIAL  AND  INTELLECTUAL  CAPITAL 


Social  and  Intellectual  Capital  Present  in  a  Healthy  Professional  Forum 

Social  capital,  as  defined  by  Nahapiet  &  Ghoshal  (1998),  is  comprised  of  the  physical 
cormections  among  people,  the  shared  language  and  perspective  among  these  people,  and  the 
soeial-psychological  relations  that  these  people  share  with  one  another.  This  definition  of  social 
capital  roughly  corresponds  to  Dixon  et  al.’s  (2005)  construct  of  “connections”  among  the 
members  of  an  APF,  but  also  recognizes  the  role  of  shared  context,  trust,  and  mutual  obligations 
among  forum  members  in  facilitating  interpersonal  growth  and  the  development  of  intellectual 
capital.  Intellectual  capital  comprises  the  knowledge  that  is  exchanged  and  constructed  among 
the  members  of  an  APF.  This  definition  of  intellectual  capital  corresponds  roughly  to  Dixon  et 
al.’s  (2005)  construct  of  “content.”  A  further  description  of  each  aspect  of  social  and  intellectual 
capital  is  provided  below. 


Connections 

Connections  are  the  overall  structure  of  who  is  reached  and  how  (Nahapiet  &  Ghoshal, 
1998).  Dixon  et  al  (2005)  point  out  that  in  an  APF,  those  who  need  to  know  must  be  cormected 
to  those  who  know.  These  individuals  comprise  the  target  membership  of  the  community,  whose 
scope  must  be  carefully  defined  in  order  to  keep  member  interaction  relevant  and  engaging 
(Gerber,  2003;  Stuckey  &  Smith,  2004).  Similarly,  a  balance  of  people  who  know  and  who  need 
to  know  among  the  forum’s  actual  members  is  important  for  facilitating  productive  discussion. 
The  connections  between  individuals  assembled  in  this  way  support  the  exchange  and  integration 
of  knowledge  necessary  for  the  development  of  intellectual  and  social  capital  to  occur  (Nahapiet 
&  Ghoshal,  1998). 

Connections  among  members  of  an  APF  may  take  the  form  of  discussion  threads  inside 
the  forum  or  emails,  phone  calls,  or  face-to-face  conversations  outside  the  forum.  Dixon  et  al. 
(2005)  describe  the  important  role  that  APF  facilitators  play  in  helping  forum  members  establish 
these  connections.  When  individuals  post  a  question  in  the  forum,  facilitators  should  respond 
immediately  to  the  post,  indicating  to  the  individual  that  their  question  has  been  acknowledged. 
Facilitators  then  use  their  own  connections  or  contact  members  of  the  forum  to  identify  the  best 
person  to  answer  the  question  in  a  timely  fashion.  The  swiftness  with  which  networks  can  be 
established  via  the  forum  is  a  function,  therefore,  of  the  depth  and  breadth  of  the  facilitators’ 
social  networks  and  how  easily  the  background  and  expertise  of  forum  members  can  be 
determined.  Sufficient  background  information  provided  in  member  bios  also  allows  forum 
members  to  be  proactive  in  seeking  cormections  with  experts.  Because  responding  thoughtfully 
to  questions  about  the  profession  takes  time  and  effort,  the  experts  contacted  to  make  a 
contribution  must  be  motivated  to  serve  the  individual  who  contacted  them,  the  forum  of  which 
the  individual  is  a  member,  or  the  professional  community  more  broadly.  The  degree  to  which 
forum  members  fill  out  their  member  bios  may  reflect  one’s  self-perception  as  a  contributor  to 
the  community  or  willingness  to  be  identified  as  an  expert.  For  example,  very  high  ranking 
officers  who  wish  to  learn  from  the  community  may  not  fill  out  biographical  information  out  of 
concern  that  their  recognized  presence  in  the  forum  may  stifle  conversation  and  activity. 
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Motivation  to  participate  in  knowledge  exchange  requires  (a)  trust  that  the  exchange  will 
not  lead  to  undesirable  professional  consequences  (Gerber,  2003);  (b)  expectation  that  the 
exchange  will  be  conducted  in  a  professional  manner  according  to  shared  values;  (c)  a  sense  of 
obligation  to  serving  the  community  present  in  the  forum;  and  (d)  personal  identification  as  a 
member  of  the  forum  (Nahapiet  &  Goshal,  1998).  For  this  reason,  the  APF  must  not  only 
facilitate  connections  between  people  on  the  basis  of  the  knowledge  they  can  impart  but  also  on 
the  basis  of  shared  norms,  values,  and  expectations.  The  norms  and  values  established  by  tone  of 
the  forum  and  maintained  through  monitoring  and  facilitation  help  to  develop  a  stable,  giving 
relationship  between  members  and  the  community. 

Context 

Nahapiet  &  Ghoshal  (1998)  identified  two  aspects  of  shared  context:  shared  narratives 
and  shared  codes  and  language.  These  theorists  argue  that  shared  context  facilitates  the 
development  of  intellectual  capital  by  (a)  providing  cognitive  access  to  individuals  with 
knowledge  to  share;  (b)  fostering  the  expectation  that  knowledge  exchange  will  be  useful;  and 
(c)  enabling  the  conceptual  overlap  that  makes  knowledge  exchange  possible.  Shared  context 
may  be  facilitated  through  restrictions  on  membership  in  the  APF  and/or  it  may  grow  out  of 
activity  in  the  forum. 

Dixon  et  al.  (2005)  claim  that  a  sense  of  shared  context,  particularly  shared  narratives, 
among  APF  members  can  be  fostered  through  participation  in  shared  experiences  enabled  by  the 
forum.  These  shared  experiences  can  take  several  forms,  such  as  collaborative  book  reviews, 
synchronous  group  discussions  online  or  face-to-face,  regular  membership  seminars  or  meetings, 
or  collaborative  exercises  online.  Importantly,  participation  itself  in  the  forum  also  creates  shared 
experience.  Forum  activity  must  foster  a  sense  of  cohesion  among  members,  a  sense  of  ease  with 
intellectual  and  social  exchange,  in  order  for  relationships  to  develop  and  continued  knowledge 
sharing  to  occur.  Cohesion  fostered  in  the  APF  could  play  an  important  role  in  facilitating 
integration  among  its  members  outside  the  forum,  for  example  if  their  duty  position  calls  them  to 
work  together  (Sanders,  2002). 


Content 

Intellectual  capital  can  take  four  forms  (Spender,  1996)':  conscious,  automatic, 
objectified,  and  collective.  Conscious  knowledge  is  an  individual’s  explicit  knowledge, 
exemplified  by  his  or  her  tools  or  explicit  procedures  used  to  solve  practical  problems.  In  the 
context  of  an  APF,  such  tools  or  procedures  may  include  policy  letter  templates,  orders  formats, 
unit  SOPs,  and  training  management  matrices,  among  other  baseline  products  (Kilner,  2002,  also 
Bobrow  &  Whalen,  2002,  and  Gerber,  2003).  A  person’s  conscious  knowledge  at  times  may  be 
difficult  to  distinguish  from  his  or  her  automatie,  or  tacit  knowledge  (Spender,  1996).  Automatic 
knowledge  often  takes  the  form  of  stories— stories  of  success  or  failure  from  which  the  discerning 
learner  can  extract  the  link  between  particular  situational  contexts  and  the  appropriate  actions  to 
take  in  these  contexts  (Cianciolo  et  al.,  2004;  Spender,  1996).  Stories  may  provide  important 
illustrative  context  for  effectively  applying  another  person’s  conscious  knowledge. 
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Objectified  knowledge  is  an  organization’s  explicit  knowledge,  which,  in  the  case  of  the 
Army,  would  include  doctrine,  lessons  learned,  tactics,  techniques,  and  procedures,  and 
administrative  procedures,  among  others.  Objectified  knowledge  has  been  validated  according  to 
institutional  standards  (Spender,  1 996)  and  serves  multiple  functions,  such  as  providing  a  frame 
of  reference  for  decision  making  and  taking  action,  providing  a  means  for  holding  individuals 
accountable  for  their  behavior  in  the  organization,  and  providing  a  foundation  on  which  further 
knowledge  is  built.  Collective  knowledge  is  an  organization’s  tacit  knowledge,  implicit  rules— 
socially  generated  and  socially  observed— for  the  behavior  of  organization  members.  Collective 
knowledge  shapes  individuals’  understanding  of  organizational  expectations  for  their  behavior, 
expectations  that  may  or  may  not  be  made  explicit  in  objectified  knowledge.  In  the  case  of  the 
Army,  collective  knowledge  may  include  shared  understanding  of  what  behaviors  reflect  Army 
values,  how  individuals  in  particular  positions  should  interact  with  one  another,  and  what  actions 
are  likely  to  lead  to  success  within  the  organization. 

Spender  (1996)  claimed  that  sharing  explicit  (i.e.,  conscious  and  objectified)  knowledge 
is  akin  to  a  library  or  database  access  and  that  only  the  development  of  tacit  (i.e.,  automatic  and 
collective)  knowledge  reflects  true  learning.  This  point  is  arguable,  depending  on  whether  the 
perception  of  new  affordances  enabled  by  explicit  knowledge  is  considered  a  form  of  learning 
(Gibson  &  Pick,  2000)  or  whether  accomplishing  new  understanding  is  considered  necessary  for 
applying  new  explicit  knowledge.  In  any  case,  the  APF  must  support  the  exchange  and 
application  of  explicit  knowledge  as  well  as  the  sharing  and  generation  of  tacit  knowledge,  as 
both  are  critical  determinants  of  performance. 

The  question  of  what  knowledge  should  be  present  in  an  APF  and  how  to  judge  its 
quality  may  not  be  quite  so  difficult  as  it  seems  initially.  Simply  put,  the  applicability  of 
knowledge  indicates  its  value.  Spender  (1996)  strongly  resisted  the  “objectification”  of 
knowledge,  noting  that  knowledge  must  be  interpreted  in  terms  of  its  utility  in  supporting 
purposeful  action.  Importantly,  objectification  and  assessment  of  forum  content  on  the  basis  of  a 
criterion  other  than  relevance  may  create  the  perception  that  external  interests  are  driving  the 
forum.  The  interviews  conducted  as  part  of  this  project  universally  acknowledge  that  APFs 
driven  by  the  top  of  the  organization,  and  not  by  the  members  of  the  APFs  themselves,  will  fail. 
Driving  the  APF  from  the  top  not  only  may  erode  the  trust  of  the  forum  members  but  also  may 
steer  the  content  and  conversation  away  from  their  needs  (Gerber,  2003).  Assessment  of  APF 
content  will  result  in  inadvertent  driving  from  the  top  if  assessment  criteria  are  based  on  the 
needs  of  those  at  the  higher  levels  of  the  organization,  rather  than  the  members  of  the 
community. 

Dixon  et  al.  (2005)  also  emphasized  the  importance  of  relevance  when  evaluating  the 
quality  of  knowledge.  Emphasis  on  relevance  implies  that  trained  professionals  are,  to  varying 
degrees,  experts  in  their  own  right,  capable  of  making  sound  judgments  regarding  the  quality  of 
knowledge.  The  amount  of  range  restriction  in  intellectual  ability  that  is  repeatedly  shown  in 
professional  populations  supports  the  assumption  that  most  Army  officers  will  exhibit  relatively 
sound  judgment  about  when  to  apply  knowledge  acquired  in  an  APF.  Setting  the  tone  in  the  APF 
for  a  professional  exchange  serves  as  a  partial  barrier  to  making  ineffectual  and/or  inappropriate 
posts  (including  posts  that  do  not  meet  information  assurance  requirements).  Community 
policing  by  both  facilitators  and  forum  members  aids  in  the  rapid  removal  of  inappropriate  posts. 
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Along  these  same  lines,  a  second  implication  is  that  abstract  criteria  are  insufficient  for 
judging  the  quality  of  knowledge.  Rather,  the  quality  of  knowledge  is  reflected  in  the  degree  to 
which  it  is  aligned  with  the  needs  of  the  forum  member  seeking  it.  Knowledge  will  be  maximally 
relevant,  and  thus  of  highest  quality,  if  it  is  developed  by  the  target  members  of  the  forum,  rather 
than  by  other  interested  parties^.  To  the  degree  that  the  information  needs  of  forum  members  are 
aligned  with  key  determinants  of  organizational  effectiveness,  knowledge  sharing  and 
development  will  have  a  powerful  impact  on  the  organization. 

Dixon  et  al.  (2005)  pointed  out  that  the  relevance  of  knowledge  is  determined  in  part  by 
the  degree  to  which  it  can  be  directly  applied  as  opposed  to  modified  before  application. 

Although  this  criterion  is  clearly  sensible,  nearly  all  knowledge  shared  or  developed  in  an  APF 
will  require  some  degree  of  intelligent  translation  for  application  to  a  specific  context.  Forum 
members  who  make  contributions  should  provide  contextual  information  that  will  aid  in  the 
interpretation  of  their  posts.  In  addition,  facilitators  should  be  actively  engaged  in  ensuring  that 
context  is  provided,  for  example  through  establishing  connections  among  posts  such  that  the 
same  knowledge  discussed  in  multiple  contexts  can  be  revealed  and  understood.  Such 
connections  and  synthesis  expedite  knowledge  sharing  and  generation. 

A  final,  additional  aspect  of  relevance  is  timeliness  (Dixon  et  al.,  2005).  Content  in  the 
forum  must  be  continually  updated  in  response  to  the  needs  of  its  members,  with  new  additions 
made  readily  apparent.  To  some  extent,  the  active  forum  serves  as  its  own  engine,  although 
facilitators  play  a  critical  role  in  keeping  discussion  active  by  sustaining  social  connections  in  the 
forum,  maintaining  meaningful  boundaries  around  the  community,  and  continually  drawing 
useful  information  from  the  environment  (Stuckey  &  Smith,  2004).  Facilitators  help  ensure  that 
questions  posted  are  answered  as  quickly  as  possible  and  that  stale  content  is  moved  from  the 
active  forum  to  an  archive. 


Conversation 

Conversations  are  the  means  by  which  a  sense  of  trust  and  community  develop  and  by 
which  new  knowledge  is  generated  (Dixon  et  al.,  2005).  Therefore,  the  APF  is  one  means  by 
which  social  and  intellectual  capital  co-evolve  (Nahapiet  &  Goshal,  1998).  Initial  intellectual 
capital  fosters  the  development  of  social  capital,  as  conversation  around  sharing  knowledge 
builds  trust  and  a  sense  of  community.  Without  conversation,  an  APF  would  simply  be  a 
searchable  database  of  information.  It  is  therefore  critical  that  conversation  be  part  of  an  APF  in 
order  for  it  to  be  considered  a  community  and  a  source  for  the  development  of  new  knowledge. 
Conversation  may  serve  different  purposes  in  the  forum,  including  clarification  or  concretization 
of  an  abstract  concept,  exploration  or  brainstorming  of  new  ideas,  challenge  or  debate  about 
existing  ideas,  or  validation  of  or  agreement  on  existing  ideas.  The  sharing  of  tacit  knowledge  is 
another  important  purpose  of  conversation  in  the  APF  often  achieved  through  storytelling. 

As  discussed  previously  with  regard  to  content,  conversations  must  be  timely,  applicable, 
and  professional  in  order  to  be  useful,  and  they  must  meet  information  assurance  requirements. 


^  However,  facilitators  do  play  an  important  role  in  establishing  the  social  connections  that  lead  to  the  sharing  and 
development  of  relevant  knowledge. 
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They  must  be  on-topic  and  develop  at  a  pace  that  suits  the  needs  of  the  person  who  started  the 
conversation.  Importantly,  conversations  must  concretize  abstract  ideas  by  providing  context  that 
will  aid  in  the  application  of  the  knowledge  they  contain.  Discussion  threads  should  build 
meaningfully  on  one  another,  resulting  in  synthesized  ideas  or  lessons  learned.  Ideally, 
discussion  threads  are  driven  by  the  target  members  of  the  community  themselves  so  as  to 
prevent  “discussion  stray,”  which  reduces  the  utility  of  the  conversation  for  the  forum  members 
(Major  Tony  Burgess,  personal  communication).  Because  conversations  are  a  shared  experience 
among  forum  members,  they  must  build  trust  and  a  sense  of  community  through  adherence  to  the 
values  of  the  forum  and  the  standards  of  professionalism.  Conversations  that  build  trust  are 
respectful  and  constructive  and  do  not  digress  into  strings  of  complaints  or  personal  attacks. 
Conversations  that  lead  to  mutual  understanding  and  knowledge  generation  build  a  sense  of  trust 
that  participating  in  the  community  will  be  helpfiil  and  boost  motivation  to  actively  participate. 

Informal,  face-to-face  communities  have  long  provided  opportunities  to  generate 
knowledge  through  discussion.  These  groups  are  more  limited  in  their  size  and  reach  than  the 
APF  because  members  must  come  from  a  relatively  small  geographical  area.  Use  of  phone  and 
email  increases  the  scope  of  these  groups,  but  not  to  the  extent  made  possible  by  the  online 
forum.  However,  building  an  online  meeting  space  is  no  more  the  same  as  building  a  healthy 
forum  than  outfitting  a  classroom  is  the  same  as  conducting  an  effective  class.  Activity  in  the 
APF  is  as  much  a  function  of  having  the  right  place  to  gather  as  it  is  having  the  right  topics  to 
discuss  and  the  right  people  with  whom  to  have  conversations.  Review  of  the  structure  of  APF 
content  and  membership  provides  insight  into  how  active  that  forum  can  expect  to  be.  Analysis 
of  the  quality  of  APF  member  interaction  and  of  the  products  of  this  interaction  provides  insight 
into  how  much  of  an  impact  that  forum  can  be  expected  to  have. 

Social  and  Intellectual  Capital  -  A  Proposed  Framework  for  Assessment 

As  with  evaluating  APF  structure  and  characteristics,  a  red-amber-green  rating  system 
can  be  used  in  order  to  translate  qualitative  information  about  social  and  intellectual  capital  into 
“analytic”  metrics.  The  assessment  framework  presented  in  this  section  features  the  social  and 
intellectual  capital  elements  for  which  status  should  be  assessed,  and  a  description  of  the  rating 
criteria  (“red,”  “amber,”  and  “green”)  for  each  of  these  elements.  Overall  Status  can  be 
determined  by  taking  the  average  of  the  component  status  ratings  featured  in  each  table  if  the 
red-amber-green  scale  were  treated  like  its  analogue,  a  1-5  ordinal  rating  scale. 
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Table  4.  Assessment  Framework  for  Connections 
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Table  7.  Assessment  Framework  for  Conversation 


ASSESSING  LEADER  COMPETENCE  AND  PROFESSIONALISM 


Assessment  of  the  impaet  of  APE  activity  on  individual  leader  competence  and 
professionalism  requires  identifying  the  general-level  competencies  that  give  rise  to  effective 
performance  in  the  jobs  of  the  forum  members.  It  also  involves  identifying  how  the  Army 
defines  professionalism  as  it  relates  to  the  target  members  of  the  forum.  Army  values  and  certain 
other  personal  characteristics  specified  in  doctrine  represent  a  significant  component  of  what  it 
means  to  be  an  Army  professional  and  are  applicable  across  APFs.  The  Army’s  overarching 
framework  of  “Be,  Know,  Do”  (FM  22-100  -  Leadership^)  captures  what  it  means  to  be  an 
Army  professional  in  terms  of  personal  characteristics,  competence,  and  action. 

In  order  to  manage  the  scope  of  the  impact  assessment,  one  should  target  those  personal 
characteristics,  competencies,  and  actions  for  which  common  or  persistent  problems  have  been 
identified  in  the  target  membership.  These  are  the  areas  where  one  can  expect  to  see 
improvement  due  to  an  intervention  because  performance  is  not  already  at  ceiling.  These  are  also 
the  areas  where  performance  improvement  is  most  badly  needed,  so  impact  in  these  areas  is  most 
likely  to  be  aligned  with  organizational  objectives. 

There  are  multiple  ways  to  assess  whether  APF  activity  has  had  an  impact  on  the 
personal  characteristics,  competencies,  and  actions  selected.  Ideally,  one  would  develop  and 
administer  in  a  quasi-experimental  setting  valid,  reliable  special-purpose  assessments  to  capture 
leader  competence  and  professionalism  both  before  and  after  becoming  a  member  of  an  APF. 
Growth  in  these  areas  could  be  compared  to  the  growth  occurring  in  these  same  areas  for  APF 
non-members  with  otherwise  similar  characteristics  (e.g.,  years  in  the  Army,  participation  in 
institutional  instruction,  etc.). 

Given  that  leaders  do  not  typically  become  members  of  an  APF  en  masse,  and  that  the 
construction  and  validation  of  the  required  special-purpose  measures  would  likely  necessitate  a 
nontrivial  research  effort  of  its  own,  the  feasibility  of  the  quasi-experimental  approach  is  highly 
questionable.  A  reasonable  proxy  is  to  assess  whether  or  not  the  content  and  discussion  in  the 
APF  addresses  the  key  problem  areas  in  leader  competence  and  professionalism  identified  in  the 
initial  analysis.  Using  the  social  and  intellectual  capital  assessment  criteria  described  in  the 
previous  section  of  this  report,  APF  functional  effectiveness  in  the  key  areas  of  interest  could  be 
assessed.  For  assessing  aspects  of  community  development  and  professionalism,  it  would  be 
feasible  to  survey  APF  members  and  non-members,  asking  them  to  provide  self-reports  of 
community  and  professional  identity.  Comparing  classroom  performance  data  and/or  officer 
evaluation  report  data  between  APF  members  and  non-members  may  serve  as  a  substitute  for 
special-purpose  measures  and  surveys,  provided  there  is  not  significant  range  restriction  in 
grades  and  evaluations.  If  comparisons  between  members  and  non-members  are  used,  it  is 
important  to  distinguish  between  more  and  less  active  members  so  that  average  levels  of  impact 
can  be  understood  in  terms  of  time  spent  in  the  forum. 


3 

FM  22-100  is  currently  under  revision  and  will  be  reissued  as  FM  6-22.  Many  of  the  concepts  and  the  underlying 
structure  presented  in  FM  22-100  will  carry  forward  to  FM  6-22,  so  much  of  the  framework  described  in  this  report 
will  remain  intact  after  the  revised  doctrine  is  issued. 
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ASSESSING  ORGANIZATIONAL  EFFECTIVENESS  -  UNIT 


To  identify  targets  for  impact  assessment  at  the  unit  level,  “unit”  must  be  defined  in  terms 
of  the  APF  members.  For  example,  if  the  APF  is  geared  toward  company  commanders,  the  unit 
should  be  defined  as  a  company  with  special  emphasis  on  the  interaction  between  the  company 
commander  and  his  or  her  platoon  leaders.  If  the  APF  is  geared  towards  staff  executive  officers, 
the  unit  should  be  defined  as  the  staff.  Defining  “unit”  in  this  way  ensures  that  unit  performance 
falls  largely  within  the  sphere  of  influence  of  the  forum  member  rather  than  external  factors, 
which  is  required  for  a  fair  assessment  of  impact. 

As  with  assessing  leader  competence  and  professionalism,  key  problem  areas  in  unit 
performance  (once  “unit”  has  been  defined)  should  be  identified  in  order  to  manage  the  scope  of 
the  impact  evaluation.  Similarly,  evaluating  organizational  effectiveness  at  the  unit  level  should 
also  focus  assessment  on  these  key  problem  areas. 

Assessment  of  unit  effectiveness  ideally  involves  observation  of  the  unit  performance  of 
APF  members  and  non-members.  Unfortunately,  as  with  assessing  leader  competence  and 
professionalism,  assessing  unit  performance  would  require  the  development  and  validation  of 
unit  performance  measures,  not  to  mention  significant  logistical  and  administrative  challenges.  A 
reasonable  proxy  would  be  to  compare  the  unit  performance  of  APF  members  and  non-members 
as  assessed  by  observer/controllers  when  the  units  rotate  through  combat  training  centers  or  as 
represented  by  operational  data  from  units  recently  returned  from  deployment.  As  stated 
previously,  if  comparisons  between  members  and  non-members  are  used,  it  is  important  to 
distinguish  between  more  and  less  active  members  so  that  average  levels  of  impact  can  be 
understood  in  terms  of  time  spent  in  the  forum.  An  alternative  proxy  is  to  assess  whether  or  not 
the  content  and  discussion  in  the  APF  addresses  the  key  problem  areas  in  unit  performance 
identified  in  the  initial  analysis.  Here  too,  one  could  use  the  social  and  intellectual  capital 
assessment  criteria  to  assess  the  APF  functional  effectiveness  in  the  key  areas  of  interest. 

ASSESSING  ORGANIZATIONAL  EFFECTIVENESS  -  THE  ARMY 

In  one  sense,  identifying  targets  for  impact  assessment  at  the  organizational  level  is 
simple;  the  Army  seeks  to  reduce  costs  while  at  the  same  time  improve  capability.  These  broad 
goals  may  be  accomplished,  for  example,  by  improving  retention,  a  matter  of  critical  importance 
to  today’s  Army,  streamlining  the  transformation  effort,  and  maintaining  high  levels  of 
professionalism  in  a  time  of  great  flux  in  roles  and  responsibilities. 

The  difficulty  in  assessing  impact  at  the  organizational  level  lies  in  the  fact  that  one  APF 
can  only  reach  a  small  fraction  of  the  Army’s  leaders.  Moreover,  it  may  take  several  years  to  see 
impact  as  junior  members  of  an  APF  work  toward  positions  of  authority  in  the  organization.  It 
reasonably  may  be  expected  that  multiple  APFs  are  required  to  make  a  demonstrable  difference 
at  the  organizational  level  in  the  broadest  sense. 

That  said,  it  may  be  possible  to  capture  the  potential  organizational  impact  of  a  single 
APF  by  identifying  the  implications  of  organizational  initiatives,  such  as  transformation,  for  the 
jobs  of  the  APF  members.  Job  demands  placed  on  APF  members  by  organizational  initiatives 
should  be  addressed  in  the  forum  in  order  for  the  forum  to  have  an  impact  on  the  performance  of 
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the  organization  as  a  whole.  In  addition,  knowledge  sharing  and  discussion  in  the  forum  should 
support  continual  development  in  support  of  organizational  goals,  including  increased  sense  of 
commitment  to  the  organization  and  enhanced  capability  to  make  learning  a  lifelong  endeavor. 

ILLUSTRATION  -  APPLYING  THE  METRICS  TO  EVALUATING  S3-XO.NET 

In  order  to  test  the  applicability  and  feasibility  of  our  metrics  and  their  associated 
measures  (e.g.,  Tables  1  through  7),  we  used  them  to  assess  S3-XO.net,  an  APF  geared  towards 
battalion  and  brigade  operations  officers  (S3s)  and  executive  officers  (XOs).  Specifically,  we 
assessed  the  strength  of  S3-XO.nefs  structure  and  characteristics,  the  effectiveness  of  the  social 
and  intellectual  capital  developed  in  S3-XO.net,  and  the  potential  for  individual,  unit,  and 
organizational  impact  of  S3-XO.net  content.  The  complete  analysis  is  presented  in  Appendix  B, 
with  only  the  general  findings  summarized  here. 

The  causal  framework  was  applied  successfully  to  evaluating  S3-XO.net  in  a  cost- 
effective  and  diagnostic  manner.  At  the  time  of  our  analysis,  we  found  that  although  current  and 
future  S3s  and  XOs  were  desirous  of  a  means  to  connect  and  share  knowledge  with  experts, 
member  activity  in  S3-XO.net  was  occurring  at  lower  levels  than  was  expected  given  the  age  of 
the  forum.  Our  investigation  revealed  that  greater  emphasis  was  devoted  to  the  technical 
specification  and  maintenance  of  the  forum,  rather  than  to  the  means  by  which  activity  in  the 
forum  would  be  generated.  Moreover,  S3-XO.net  lacked  the  personnel  structure  and  support 
necessary  to  improve  the  functioning  of  the  forum.  For  these  reasons,  metrics  of  leader 
competence  and  professionalism  and  organizational  effectiveness  were  explored  but  the  attempt 
was  not  made  to  measure  them. 


CONCLUDING  REMARKS 

The  present  research  represents  an  initial  effort  to  explore  and  develop  a  general 
framework  for  assessing  the  effectiveness  and  impact  of  APFs.  We  have  based  the  framework  on 
an  overview  of  the  scholarly  and  technical  literature  on  knowledge  sharing  and  its  assessment. 
The  framework  and  its  application  are  also  based  on  interviews  with  a  range  of  people  involved 
in  APFs— from  organizational  thinkers  to  forum  members  to  forum  developers.  The  result  of  our 
initial  work  is  a  generalizable  framework  that  can  be  used  as  a  foundation  for  assessing  the 
effectiveness  and  impact  of  APFs  at  both  the  beginning  and  advanced  stages  of  their 
development.  Appendix  C  provides  some  guidance  as  to  when  assessment  should  be  conducted 
to  maximize  its  utility. 

Future  research  can  take  several  directions  to  advance  this  effort.  First,  the  validity  of  the 
APF  assessment  framework  when  applied  to  multiple  APFs  should  be  explored.  The  exploration 
should  feature  APFs  in  differing  phases  of  development,  from  very  early  stages  to  advanced 
stages.  Such  an  exploration  would  further  illuminate  the  link  between  APF  structure  and 
characteristics  and  APF  activity.  Second,  individual-,  unit-,  and  organization-level  impact  should 
be  analyzed  using  more  active  APFs,  such  as  CC.mil.  This  analysis  would  illuminate  the  link 
between  APF  activity  and  impact,  as  well  as  identify  appropriate  means  for  directly  assessing 
impact  through  special  purpose  measures  (e.g.,  surveys,  tactical  games,  etc.),  observation  of  unit 
perfonnance,  and  cohort  tracking  on  organizational  variables  of  interest  (e.g.,  retention).  Finally, 
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the  validity  of  the  red-amber-green  assessment  criteria  should  be  examined  closely  by 
systematically  investigating  the  link  between  different  levels  of  activity  in  APFs  and  different 
levels  of  individual-  and  unit-level  impact  as  directly  assessed. 

As  a  parting  comment,  we  advise  that  careful  attention  be  paid  to  ensuring  that  APF 
assessment  reflects  the  delicate  balance  between  the  requirements  of  forum  members  to  grow 
organically  in  an  informal,  consequence-free  environment  and  the  requirements  of  the 
organization  to  justify  its  investment  in  knowledge  sharing  to  stakeholders.  The  consensus  in  the 
literature  indicates  that  too  much  organizational  involvement,  perceived  or  otherwise,  will  reduce 
the  trust  of  forum  members  and  ultimately  kill  the  informal  knowledge  sharing  that  provides 
organizational  advantage  (e.g.,  Gerber,  2003;  Wenger  &  Snyder,  2000).  However,  knowledge¬ 
sharing  forums  depend  on  organizational  cultivation  to  thrive  and  live  up  to  their  promise 
(Wenger  &  Snyder,  2000).  Successful  effectiveness  and  impact  assessment  therefore  must  serve 
to  direct  cultivation  rather  than  impose  requirements  or  change. 

Consider  the  following  analogy,  extended  from  Wenger  and  Snyder  (2000).  A  farmer  one 
day  discovers  that  wild  onions  are  growing  in  one  of  his  plots.  He  shares  a  couple  of  these  onions 
with  some  close  friends,  and  everyone  agrees  they  are  quite  unique  and  flavorful.  He  believes 
that  cultivating  and  selling  the  onions  will  give  his  farm  a  competitive  advantage.  However,  the 
onions  are  easily  pushed  out  by  other  plants  and  susceptible  to  dry  spells.  Motivated  by  the 
payoff  he  thinks  cultivating  the  onions  will  provide,  he  invests  time  and  money  to  clear  a  plot  for 
them  (reducing  the  plot  size  of  some  other  vegetable)  and  to  set  up  a  rudimentary  irrigation 
system.  The  onions  thrive.  The  farmer  is  pleased,  but  he  knows  the  real  test  of  how  wisely  his 
resources  were  spent  is  whether  the  onions  sell  at  the  farmer’s  market.  He  uses  his  onion  sales  at 
the  farmer’s  market  as  an  assessment  of  his  judgment.  Naturally,  the  onion  sales  fluctuate  as 
market  demand  changes.  When  sales  are  up,  he  continues  his  current  cultivation  practices.  He 
uses  quiet  sales  periods  as  evidence  that  he  needs  to  change  something  in  his  cultivation 
practices.  He  changes  techniques  to  help  bring  out  the  best  the  onions  have  to  offer,  such  as 
sweet  flavor  one  year  and  size  another  year.  The  onions  thrive,  the  market  responds,  and  the 
farmer  is  pleased. 

In  the  present  research,  it  was  our  intention  to  lay  the  groundwork  for  an  assessment 
framework  that  directs  APF  cultivation.  The  application  of  our  framework  to  assessing  the 
effectiveness  and  impact  of  S3-XO.net  demonstrates  how  assessment  can  lead  to 
recommendations  for  better  serving  the  needs  of  the  forum  members  and,  by  extension,  the  needs 
of  the  Army.  Future  research  must  follow  this  lead  such  that  APF  assessment  stimulates  growth, 
rather  than  stifles  it. 
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APPENDIX  A  -  LIST  OF  INTERVIEWEES 


CC.mil  Core  Group  Members  (3) 
NCOTeam.org  Faeilitators  (2) 
lnstructor.net  Developer  (1) 

S3-XO.net  Founding  Offieers  (3) 
S3-XO.net  Facilitator/ Administrators  (5) 
NTC  Observer/Controllers  (8) 

Future  S3s/XOs  (7) 

Current  or  Past  S3s/XOs  (2) 

Instructors  (3) 

Doctrine  Writers  (2) 

Organizational  Thinkers  (5) 
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APPENDIX  B  -  ASSESSMENT  OF  S3-XO.NET 


In  the  present  research,  the  metrics  and  assessment  frameworks  described  in  the  main 
body  of  this  report  were  applied  to  evaluating  the  Army  Professional  Forum  (APF)  S3-XO.net. 
This  section  presents  the  results  of  the  assessment  and  recommendations  for  enhancing  the 
forum’s  activity  and  potential  impact. 

A  Brief  History  of  S3-XO.net 

S3-XO.net  was  designed  to:  “Create  and  foster  knowledge  and  experience  sharing  among 
future,  serving,  and  prior  operations  [officers]  and  executive  officers  [S3s  and  XOs,  at  the 
battalion  and  brigade  level]  in  order  to  build  combat  ready  units  {S3-XO.Net  In-Process  Review 
#2,  29,  as  cited  in  Gerber,  2003).”  S3-XO.net  was  developed  during  the  winter  and  spring  of 
2003  by  a  group  of  Command  and  General  Staff  College  (CGSC)  students  under  the  tutelage  of 
Major  Tony  Burgess  (member  of  the  CC.mil  core  group)  and  Lieutenant  Colonel  Mike  Prevou 
(CGSC  professor),  and  with  the  support  of  the  then  head  of  the  Combined  Arms  Center. 

S3-XO.net  was  developed  as  part  of  an  Army  response  to  the  success  of  CC.mil,  an  APF 
launched  two  years  earlier  (Gerber,  2003).  The  Battle  Command  Knowledge  System  (BCKS)  is 
currently  under  development  as  the  main  component  of  this  response.  The  purpose  of  BCKS  is  to 
harness  knowledge  sharing  in  the  service  of  speeding  up  the  combat  decision  cycle  and  of 
enhancing  leader  professional  development.  S3-XO.net  is  currently  situated  as  one  of  several 
APFs  under  the  umbrella  of  the  Leader  Network  component  of  BCKS.  Whereas  other 
components  of  BCKS,  such  as  Warrior  Knowledge  Base  and  Unit  Network,  are  geared 
specifically  to  enhancing  battle  command.  Leader  Network  is  intended  to  have  its  primary 
impact  on  the  development  of  professional  competence. 

The  founding  officers  of  S3-XO.net  developed  a  plan  for  the  forum,  designed  the  online 
meeting  space,  and  generated  initial  activity  in  the  forum  after  it  was  launched.  Within  months  of 
launching  S3-XO.net,  in  May  2003,  its  founders,  including  Major  Burgess,  graduated  from 
CGSC  and  were  reassigned  (Gerber,  2003).  Activity  in  S3-XO.net  therefore  dropped  significantly 
immediately  after  its  launch  as  the  founding  officers  moved  on.  Moreover,  Lieutenant  Colonel 
Prevou  was  deployed  to  Afghanistan  for  one  year,  so  the  leadership  of  the  APF  by  necessity  was 
handed  off  in  August/September  of  2003.  Two  majors  at  Fort  Leavenworth  assumed  leadership 
of  the  forum  at  this  time.  In  the  spring  of  2004,  as  activity  on  S3-XO.net  was  regaining 
momentum,  S3-XO.net  was  shut  down  for  approximately  two  months  due  to  concerns  about  the 
security  of  the  information  discussed  on  the  site.  After  placing  login  restrictions  [i.e.,  use  of 
Army  Knowledge  Online  (AKO)  username  and  password]  and  acquiring  permission  to  restart, 
the  S3-XO.net  leadership  brought  the  site  back  up  in  May,  2004.  In  December  of  2004,  due  to 
difficulties  re-establishing  momentum'*  on  the  now-secure  site,  the  community  was  re-launched 
in  an  effort  to  get  the  word  out  that  S3-XO.net  was  indeed  back  up  and  running. 


This  difficulty  re-establishing  momentum  may  have  been  due  to  several  causes  including  lack  of  awareness  that  the 
site  had  been  re-launched  and  difficulty  with  the  new  login  procedures,  but  it  is  also  likely  due  to  the  reassignment 
of  all  but  one  of  the  founding  officers  without  a  succeeding  core  group  evolving  out  of  the  peripheral  members  of 
the  forum. 
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The  leadership  of  S3-XO.net  was  handed  off  a  second  time  in  January  2005,  in  an  effort 
to  institutionalize  and  stabilize  the  forum’s  foundation.  The  head  of  Fort  Leavenworth’s  Center 
of  Army  Tactics  has  been  assigned  to  oversee  the  community’s  progress,  with  a  CGSC  instructor 
serving  as  his  action  officer.  CGSC  instructors  currently  serve  as  part-time  facilitators  and  one 
on-site  contractor  provides  full-time  technical  and  editorial  support.  Through  a  course 
requirement,  CGSC  students  also  play  a  facilitation  role  by  participating  in  discussion  threads. 
S3-XO.net  in  particular,  and  knowledge  management  in  general,  has  the  full  support  of  the 
commander  of  the  Combined  Arms  Center. 


As  of  2003,  there  were  approximately  23,545  personnel  in  the  S3-XO.net  Regular  Army 
target  audience—  approximately  16,205  majors  and  lieutenant  colonels  and  7,340  captains  on  the 
career  path  that  involves  S3/XO  positions  (Gerber,  2003).  Including  Marine  Corps  or  Reserve 
component  personnel  in  the  S3-XO.net  target  audience  would  approximately  double  its  size 
(Gerber,  2003).  Figure  2  depicts  a  screen  capture  of  S3-XO.net' s  main  page  as  it  currently  looks. 
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Welcome  aboard  new  members.  We  have  grown!  We  currently  boast  roughly 
2500  future,  current,  and  past  S3s  and  XOs.  Members  have  posted  5300 
H  knowledge  objects  on  our  site.  Our  content  continues  to  grow  creating  a  vast 
knowledge-base  for  our  users. 

S3s  and  XOs  are  experts  that  are  distributed  across  the  Army  and  often 
disconnected  from  their  peers.  S3-XONet  is  your  antidote  to  this  problem.  We 
vv.-.i  promote  a  new  method  of  collaboration,  information  exchange,  and  sharing  of 
I  best  practices  across  boundaries  of  time  and  distance.  You  will  be  able  to 
tXii  contribute  to  the  generation  of  new  ideas  and  make  valuable  contacts.  In 
addition,  you  will  solve  daily  problems  and  foster  individual  competencies, 
making  a  significant  contribution  to  overall  Army  learning.  Check  out  Net 
Call  June  2005  to  get  the  latest  news  on  our  portal. 

This  site  is  authorized  to  carry  only  unclassified  FOUO  information  and  below  only.  (FOUO)  Tactics,  Techniques,  and 
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Figure  2.  S3-XO.net  Main  Page 
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S3-X0.net  Structure  and  Characteristics 


The  Plan 

On  5  February,  2003,  a  draft  list  of  requirements  for  S3-XO.net  was  prepared  by  one  of 
the  S3-XO.net  founding  officers.  This  list  of  requirements  has  been  slightly  modified  for  use  as  a 
list  of  requirements  for  all  of  the  APFs  in  the  BCKS  Leader  Network.  In  addition,  a  BCKS 
document  (2005)  detailing  how  to  establish  a  forum  on  the  Leader  Network  provides  additional 
administrative  requirements  (particularly  personnel  requirements)  that  apply  to  S3-XO.net.  The 
status  of  each  component  of  the  S3-XO.net  plan  is  indicated  in  Table  8,  with  a  justification  for 
each  status  rating  provided  below. 


Table  8.  S3-XO.net  Plan  Stal 

/us 

Plan  Component 

Status 

Red 

Amber 

Green 

Statement  of  Purpose 

mi 

Envisioned  Impaet 

■■ 

Description  of  Target  Audience 

Statement  of  Cultural  Norms,  Core  Values,  or  Conventions 

Specifieation  of  Roles  and  Responsibilities 

Functional  and  Technical  Specification 

a 

. 

Overall  Rating 

Statement  of  Purpose 

S3-XO. net's  statement  of  purpose  states  that  S3-XO.net  will:  “Facilitate  the  effectiveness 
of  battalion/brigade  executive  and  operations  officers. . .”  and  that  it  will  capitalize  “on  officer’s 
untapped  stores  of  experience  and  knowledge  helping  leaders  turn  tacit  knowledge  into  explicit 
knowledge  and  increase  the  learning  curve  of  all  those  participating  in  the  community.” 

Although  not  stated  explicitly,  it  appears  in  this  statement  of  purpose  that  S3-XO.net  is  intended 
to  be  a  helping  forum  because  of  its  focus  on  disseminating  tacit  knowledge,  an  important 
determinant  of  practical  problem  solving  (Wagner  &  Sternberg,  1985). 

The  implied  target  audience  is  Army  officers  who  are  either  on  the  track  to  become  a 
battalion  (BN)^rigade  (BDE)  XO  or  S3,  or  are  currently  a  BN/BDE  XO  or  S3^.  This  may 
include  captains,  majors,  and  lieutenant  colonels  in  the  Active  Component  [approximately 
23,545  personnel  (Gerber,  2003),  see  above]  and  potentially  these  same  officers  in  the  Reserve 
Component  (Gerber,  2003).  The  statement  of  purpose  also  implies  that  the  content  featured  in  the 
site  should  take  the  form  of  both  discussion  threads  and  practical  products  in  order  to  facilitate 
tacit-knowledge  transfer  and  articulation  as  explicit  knowledge  (this  is  described  explicitly  in 


^  It  may  be  argued  that  previous  BN/BDE  S3s  or  XOs,  instructors,  training  developers,  and  doctrine  writers  are  also 
members  of  the  target  audience  because  participation  in  S3-XO.net  may  support  these  individuals’  efforts  to  enhance 
S3/XO  effectiveness.  However,  to  the  degree  that  the  forum  is  designed  to  address  the  needs  of  S3s/XOs,  including 
these  other  people  in  the  target  audience  (as  opposed  to  playing  supporting  roles)  may  result  in  applying  top-down 
structure  to  the  forum  or  in  defining  the  target  audience  too  broadly,  both  of  which  will  stifle  activity  (Gerber,  2003; 
Stuckey  &  Smith,  2004;  Wenger  et  ah,  2002). 
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other  parts  of  the  requirements  doeument).  Moreover,  the  general  funetional  requirements  of  the 
system  must  inelude  a  way  for  forum  members  to  download  and  upload  files,  to  find  each  other, 
and  to  carry  on  conversation. 

The  statement  of  purpose  was  assigned  a  combined  amber/green  rating  because  although 
the  statement  has  clear  implications  for  target  audience,  it  is  less  explicit  about  what  type  of 
forum  S3-XO.net  is  intended  to  be  or  what  audience  it  is  intended  to  serve  (see  the  next  section 
for  additional  justification).  This  characterization  is  a  critical  one  to  make  (Wenger  et  ah,  2002). 

Envisioned  Impact 

The  S3-XO.net  requirements  document  makes  the  following  statements  about  envisioned 
impact:  “...  [S3-XO.net]  will  become  a  powerful  tool  in  developing  adaptive  leaders,  peer-to- 
peer  knowledge  transfer,  relevant  doctrine,  and  officers  who  are  life  long  learners.  There  is  also 
evidence  to  show  that  participation  . . .  [will]  increase  commitment  to  the  organization  and  Army 
service  ethic.”  A  list  of  potential  areas  of  impact  is  also  provided,  and  is  reproduced  below: 

•  “Foster  innovation  by  encouraging  the  free  flow  of  ideas  and  challenge  status  quo  thinking 

•  Encourage  life  long  leader  learning 

•  Improve  time  management  by  streamlining  research  and  development  of  manuals  regulations, 

SOP,  etc, 

•  Streamline  operations  and  reduce  costs  by  eliminating  redundant  or  unnecessary  processes 

•  Provide  a  sense  of  community,  worldwide,  among  leaders  stationed  in  the  U.S.  and  deployed  or 
stationed  overseas 

•  Receive  real-time  feedback  between  institutional  Army  and  field  units 

•  Sharing  of  expert  experience  and  wisdom  (tacit  knowledge) 

•  Converting  expert  tacit  knowledge  and  experiences  (wisdom)  to  explicit  knowledge 

•  Capturing  of  “actionable”  knowledge  from  lessons  learned,  operational  experiences,  projects,  etc. 
that  can  be  easily  retrieved  by  interested  parties 

•  Improving  the  quality  of  instructors  by  directly  impacting  the  experience  curve 

•  Providing  a  mechanism  to  increase  peer  to  peer  mentoring 

•  Creating  a  platform  for  sharing  knowledge  that  starts  in  the  classroom  and  can  be  extended  into 
field  operations” 

This  statement  of  envisioned  impact  provides  a  clear  description  of  how  the  effectiveness 
of  the  target  audience  is  to  be  achieved  through  participating  in  the  forum.  However,  the 
statement  does  not  describe  where  the  impact  will  be  (e.g.,  effectiveness  in  terms  of  the 
competencies  and  duties/responsibilities  of  the  S3/XO  are  not  defined)  and  therefore  does  not 
provide  criteria  for  assessing  this  impact.  Moreover,  the  impact  statement  also  suggests  that  S3- 
XO.net  is  intended  to  be  multiple  types  of  forum  at  once,  including  helping,  innovating,  and 
knowledge  stewarding,  and  is  intended  to  serve  multiple  stakeholders.  This  implies  that  previous 
BN/BDE  S3s  or  XOs,  instructors,  training  developers,  and  doctrine  writers  may  be  considered 
(implicitly  or  explicitly)  as  additional  members  of  the  target  audience  rather  than  as  playing  a 
supporting  role  in  the  forum  or  indirectly  benefiting  from  the  forum’s  activity.  Potentially 
including  these  individuals  in  the  target  audience  imposes  a  top-down  structure  to  the  content 
and  discussion  in  the  forum,  which  stifles  forum  activity  (Gerber,  2003;  Stuckey  &  Smith,  2004; 
Wenger  et  al.,  2002).  For  these  reasons,  the  envisioned  impact  was  given  a  combined  red/amber 
rating. 
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Description  of  the  Target  Audience 

Neither  the  S3-X0.net  requirements  document  nor  the  BCKS  (2005)  document  includes  a 
description  of  the  needs  of  the  target  audience.  Description  of  the  target  audience  was  therefore 
assigned  a  red  rating. 

Statement  of  Cultural  Norms,  Core  Values,  and  Conventions 

The  S3-XO.net  requirements  document  does  not  include  a  description  of  the  cultural 
norms,  core  values,  or  conventions  to  be  honored  while  participating  in  S3-XO.net.  However,  the 
BCKS  (2005)  document  describes  some  conventions,  including  presenting  only  For  Official  Use 
Only  and  above  (i.e.,  less  restricted)  information  in  the  forum,  not  using  the  forum  for  promoting 
personal  agendas  or  marketing  commercial  items,  and  keeping  conversation  professional  (e.g., 
avoiding  personal  attacks  and  other  negative  behavior).  While  conventions  are  discussed,  the 
values  to  be  upheld  in  the  forum  are  not,  nor  are  the  means  for  enforcing  adherence  to 
conventions  and  values.  For  this  reason,  the  characterization  of  cultural  norms,  core  values,  and 
conventions  has  been  assigned  a  combined  amber/green  rating. 

Specification  of  Roles  and  Responsibilities 

The  initial  S3-XO.net  requirements  document  contains  a  brief,  general  specification  of  the 
roles  and  responsibilities  of  the  forum  support  personnel.  However,  BCKS  (2005)  does  detail 
roles  and  responsibilities.  These  include  forum  members,  the  leadership  team,  the  chief  editor, 
topic  leads  and  point  leads  (i.e.,  facilitators),  the  administrative  support  team,  and  the  sponsor. 
Specific  duties/responsibilities  are  outlined  for  each  role,  as  well  as  some  guidance  for  how  the 
roles  interact/overlap.  One  shortfall  in  the  otherwise  exemplary  specification  of  the  roles  and 
responsibilities  is  the  apparent  lack  of  a  role  for  technical  leadership  and  implementation.  The 
Chief  Editor  is  assigned  some  degree  of  responsibility  over  technical  maintenance,  and  a 
software  administrator  is  mentioned,  but  additional  detail  as  to  the  interplay  between  technical 
implementation  and  forum  evolution  is  not  provided.  In  addition,  the  role  of  a  core  group  is  not 
specified.  That  is,  although  the  roles  of  the  leadership  team  described  in  BCKS  (2005)  seem 
similar  to  the  roles  of  the  core  group,  the  importance  of  a  passionate,  dedicated,  closely  knit  team 
to  support  the  forum  through  informal  leadership  does  not  seem  to  be  recognized.  Because  of  the 
importance  of  both  technical  support  and  a  passionate  core  group  to  forum  success,  the 
specification  of  roles  and  responsibilities  has  been  given  a  combined  amber/green  rating.  It 
should  be  noted  that  although  this  specification  of  roles  and  responsibilities  is  rather  thorough,  it 
has  been  drafted  nearly  two  years  after  the  release  of  S3-XO.net,  so  its  impact  on  the  current 
status  of  S3-XO.net  may  be  somewhat  limited. 

Functional  and  Technical  Specification 

The  S3-XO.net  requirements  document  has  a  very  detailed  functional  and  technical 
specification  that  has  also  been  recently  updated.  This  specification  includes  conceptual  as  well 
as  technical  requirements,  to  include  the  knowledge-sharing  behaviors  that  must  be  supported 
(e.g.,  carry  on  asynchronous  and  synchronous  conversations,  add  content,  etc.),  network 
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requirements,  front-end  and  back-end  features  of  the  site,  and  seeurity  requirements.  The 
exemplary  specifieation  was  assigned  a  green  rating. 

Personnel 


Table  9.  S3-XO.net  Personnel  Status 


Personnel  Category 

Status 

Red 

Amber 

Leadership 

Core  Group 

Facilitators 

Technical  Support 

Overall  Status 

ZZH 

Leadership 

Financial  support  for  S3-XO.net  is  channeled  through  funding  designated  for  BCKS  as  a 
whole.  Initial  funding  for  BCKS  came  in  the  summer  of  2004  from  the  Commanding  General  of 
I  Corps  in  Iraq.  The  CG  of  the  Combined  Arms  Center  (CAC)  at  Fort  Leavenworth  has  been 
assigned  sponsorship  of  BCKS  (and  therefore,  by  extension,  S3-XO.net),  and  is  responsible  for 
procuring  additional  funding  in  the  upcoming  fiscal  years.  The  CG  of  CAC  is  supportive  of 
knowledge  management  and  of  S3-XO.net  in  particular.  From  its  inception,  S3-XO.net  was 
supported  by  the  CAC  CG,  and  continues  to  enjoy  this  support  even  after  a  change  in  general 
officers  holding  this  position.  Competition  for  additional  funding  appears  to  come  from  AKO, 
which  is  the  institutionally  recognized  Army  knowledge  management  solution. 

S3-XO.  net's  formal  leadership  team  is  comprised  of  three  individuals.  Although  all  three 
of  these  people  are  supportive  of  knowledge  management  and  horizontal  knowledge  sharing, 
none  of  them  have  assumed  the  leadership  position  voluntarily.  These  three  leaders  oversee  the 
forum’s  development,  address  problem  areas  as  they  arise,  provide  general-level  vision  for  the 
forum,  and  foster  communication  between  the  informal  forum  leaders  and  the  BCKS  sponsor. 
Leadership  of  S3-XO.net  must  compete  with  the  numerous  other  responsibilities  of  these  high- 
ranking  individuals.  Of  the  informal  leaders,  one  is  among  the  officers  who  founded  the  forum. 
Both  of  the  informal  leaders  are  passionate  about  the  professional  development  of  S3s  and  XOs, 
and  work  hard  to  bring  S3-XO.net  to  the  attention  of  the  target  audience,  to  develop  the  argument 
for  securing  support  for  the  forum,  and  to  provide  administrative  leadership  to  support  personnel. 
However,  these  two  dedicated  leaders  must  also  juggle  multiple  priorities,  and  have  limited  time 
to  devote  to  leading  S3-XO.net.  The  leadership  team  (formal  and  informal)  will  experience 
turnover  of  at  least  three  of  its  members  within  a  year  as  officers  retire  or  are  reassigned. 

The  sponsorship/leadership  of  S3-XO.net  has  been  assigned  an  amber  rating.  The 
sponsorship  and  leadership  of  the  forum  is  dedieated  and  supportive  of  the  forum’s  mission,  but 
currently  lack  the  time  and  internal  compass  required  to  foster  activity  and  growth  in  the  forum. 
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Core  Group 


Currently,  S3-X0.net  lacks  a  core  group.  At  no  point  in  the  forum’s  history  did  S3-XO.net 
have  a  group  of  core  members  who  saw  themselves  as  such  and  who  took  responsibility  for 
ensuring  the  growth  of  the  forum.  S3-XO.neCs  founding  officers  consisted  largely  of  CGSC 
students  or,  in  terms  of  the  target  audience,  future  S3s  and  XOs.  Therefore  these  officers,  while 
enthusiastic  and  skillful,  were  not  fully  representative  of  the  target  audience  and  ultimately  saw 
the  forum  as  a  professional  development  tool  for  people  other  than  themselves.  With  the  help  of 
their  instructor,  who  was  previously  an  S3  and  an  XO,  they  could  articulate  in  general  terms 
what  an  S3  or  XO  could  hope  to  gain  from  membership  in  S3-XO.net,  but  they  did  not  have  first¬ 
hand  experience  with  facing  the  day-to-day  challenges  of  the  S3/XO.  They  also  did  not  have  the 
benefit  of  having  gained  a  large  social  network  of  S3s/XOs  to  work  with  in  fully  identifying 
target-audience  needs  and  interests.  Lacking  this  network,  they  did  not  have  the  means  to  rapidly 
post  content  in  the  forum  that  would  generate  discussion  among  current  S3s/XOs  or  to  link 
members  to  one  another.  The  founding  officers  (with  one  important  exception,  who  is  currently 
on  the  leadership  team),  having  left  CGSC,  are  no  longer  actively  involved  in  S3-XO.net  and  a 
core  group  of  target  audience  members  has  not  evolved  to  take  their  place.  For  these  reasons,  the 
core  group  for  S3-XO.net  has  been  assigned  a  red  rating. 

Facilitators 

Responsibility  for  S3-XO.net  facilitation  is  not  clearly  assigned  to  particular  individuals. 
One  on-site  contractor  is  assigned  full-time  to  the  forum,  but  must  split  his  time  between 
facilitation  and  technical  support,  with  the  majority  of  his  efforts  devoted  to  technical  support. 
Three  members  of  the  leadership  team  play  some  role  in  facilitation,  as  do  CGSC  instructors  and 
a  handful  of  on-site  contractors.  These  people  volunteer  their  time  to  post  content  on  the  forum 
that  will  facilitate  discussion  and  to  guide  discussion  into  productive  directions.  While  passionate 
about  the  professional  development  of  the  S3/XO,  these  people  report  having  very  limited  time 
to  devote  to  facilitation.  Such  duties  as  performing  leadership  activities  or  preparing  course 
materials  must  take  precedence  over  monitoring  discussion  threads,  synthesizing  knowledge  on 
the  forum,  and  actively  linking  forum  members.  There  also  appears  to  be  some  confusion  as  to 
whose  initiative  is  key  to  the  success  of  S3-XO.net  —  the  leadership,  the  facilitators,  or  the 
members  themselves.  Representatives  of  each  group  seem  to  believe  a  different  group  is 
primarily  responsible  for  stimulating  forum  activity.  For  these  reasons,  S3-XO.net  facilitation  has 
been  assigned  a  red  rating. 

Technical  Support 

Technical  implementation  of  S3-XO.net  is  the  primary  responsibility  of  one  on-site 
contractor  who  is  assigned  to  the  forum  full-time  both  as  technical  support  and  as  a  facilitator. 
This  person  ensures  that  the  online  meeting  space  (including  access  rights)  is  working  properly, 
makes  changes  to  the  look  and  feel  of  the  forum,  and  resolves  technical  difficulties.  However, 
S3-XO.net  does  not  appear  to  have  a  formal  role  for  technical  leadership.  This  leadership  role  is 
necessary  to  provide  vision  for  adapting  the  structure  and  appearance  of  the  forum  to  meet  the 
changing  needs  of  the  forum  membership.  Moreover,  a  representative  of  the  on-site  contractor 
and  a  member  of  the  forum  leadership  have  both  reported  that  additional  technical  staff  (e.g.,  a 
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Web  designer)  is  required  and  that  lack  of  funding  and  personnel  has  prevented  them  from  doing 
several  development  activities  they  wanted  to  do.  For  these  reasons,  S3-XO.net  technical  support 
has  been  assigned  an  amber  rating. 


Functional  Requirements 


Table  10.  53-WQ.ne^  Functional  Requirements  Status 


Functional  Requirement  Category 

Status 

Red 

Amber 

Green 

Locating  the  Forum 

' 

Joining  the  Forum 

Tone  in  the  Forum 

Conversation  Support 

Locating  Content 

Locating  People 

u.'V.i  1 

Overall  Status 

_,-j 

Locating  the  Forum 

S3-XO.net  can  be  located  through  BCKS,  via  Google  search,  and  through  AKO.  When 
using  Google  search,  the  search  terms  “S3-XO.net,”  “S3  XO  forum,”  or  “S3  XO  knowledge” 
will  return  the  S3-XO.net  homepage  as  the  number  one  hit.  Using  the  search  terms  “S3  net,”  “XO 
net,”  or  “XO  knowledge”  will  return  the  S3-XO.net  homepage  within  the  first  page  of  hits.  Using 
the  search  terms  “XO.com,”  “XO  forum,”  “S3  forum,”  or  “S3  knowledge”  will  not  return  the  S3- 
XO.net  homepage  in  the  first  seven  pages  of  hits^.  Using  the  search  term  “S3.com”  only  returns 
one  hit,  and  it  is  not  the  S3-X0.net  homepage.  Importantly,  these  results  suggest  that  the  Google- 
using  officer  seeking  to  join  S3-X0.net  should  have  some  idea  of  what  the  forum  is  called  in 
order  to  find  it  quickly.  On  the  S3-X0.net  splash  page  available  via  Google  search,  potential 
users  can  easily  click  on  a  link  to  begin  the  registration  process. 

To  locate  S3-X0.net  via  AKO,  the  potential  member  must  first  select  the  “Knowledge 
Networks”  tab,  then  select  “Battle  Command  Knowledge  System,”  then  select  “S3-X0.net”  from 
two  possible  links.  S3-X0.net  cannot  be  found  by  entering  “S3-X0.net”  or  related  terms  in  the 
AKO  search  functionality.  Here  too,  the  implication  is  that  the  potential  member  must  already 
have  some  idea  of  what  they  are  looking  for  before  they  conduct  their  search.  In  the  case  of 
AKO,  the  potential  member  must  also  know  that  S3-XO.net  is  part  of  the  BCKS,  which  is  part  of 
the  Army  Knowledge  Network.  Given  our  interviews  with  current  and  potential  forum  users,  it  is 
highly  unlikely  that  the  potential  S3-X0.net  user  will  have  this  knowledge  unless  someone  from 
the  forum  has  personally  invited  them  to  join^.  The  S3-X0.net  splash  page  available  via  Google 


^  The  use  of  “seven  pages”  as  a  rough  metric  reflects  the  assumption  that  most  people  will  have  stopped  searching 
and  decided  to  use  a  new  search  term  by  the  time  they  go  through  five  or  more  pages  of  hits  without  finding  what 
they  were  looking  for. 

’  Most  of  the  officers  we  interviewed  had  not  heard  of  the  terms  “community  of  practice”  or  “professional  forum.” 
All  of  them,  however,  knew  what  CC.mil  was.  This  suggests  that  “front-end”  terminology  (i.e.,  the  forum  name) 
should  be  used  to  aid  potential  members  in  locating  the  forum,  rather  than  “back-end”  terminology  (e.g.,  the  Army’s 
terms  or  theoretical  terms). 
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search  directs  current  users  to  login  to  the  forum  via  AKO,  but  does  not  provide  direction  as  to 
how  to  find  the  forum  once  inside  AKO.  In  fact,  the  splash  page  says  that  logging  in  to  AKO  will 
allow  the  member  to  “proceed  directly  to  the  site.”  Clicking  on  the  link  to  begin  the  registration 
process  will  allow  current  members  to  login,  but  this  is  not  evident  from  the  instructions 
provided  on  the  splash  page. 

An  important  barrier  to  word-of-mouth  advertising  for  S3-XO.net  is  that  there  are 
relatively  few  S3s  and  XOs  in  a  particular  unit  (e.g.,  approx  3  S3s  and  3  XOs  in  a  BDE,  as 
opposed  to  approximately  9  company  commanders  and  27  platoon  leaders),  who  may  or  may  not 
communicate  or  may  see  themselves  as  in  competition  with  one  another.  The  S3/XO  population 
is  therefore  widely  distributed  and  does  not  necessarily  have  the  impetus  to  communicate 
amongst  its  members.  It  is  especially  important,  then,  for  the  S3-XO.net  leadership  to  be  active  in 
getting  the  word  out  about  the  forum.  In  August  2005,  the  S3-XO.net  staff  plans  to  hold  a 
membership  drive  geared  towards  students  at  CGSC.  In  addition,  an  invitation  to  join  S3-XO.net 
will  be  sent  to  all  Army  majors.  These  efforts  represent  the  first  large-scale  initiative  to  recruit 
new  members.  For  these  reasons,  Locating  the  Forum  is  assigned  an  amber  status. 

Joining  the  Forum 

To  join  S3-XO.net,  prospective  members  must  fill  out  a  short  online  application  and  wait 
for  approval.  The  on-site  contractor  assigned  to  the  forum  checks  for  membership  applications 
on  a  daily  basis  and  grants  approval  within  24  hours. 

Four  survey  questions  (complete  survey  is  located  in  Appendix  D)  asked  S3-XO.net 
members  to  indicate  their  impressions  of  joining  the  forum.  These  questions  addressed  the 
appropriateness  of  membership  restrictions,  the  usability  of  the  membership  application,  the  time 
it  took  for  membership  to  be  granted,  and  the  tone  of  the  welcome  letter  sent  out  to  new 
members.  Seven  members  completed  the  survey  questions  out  of  19  members  to  whom  the 
survey  was  administered,  comprising  a  response  rate  of  37%^  On  average  (i.e.,  across  the  four 
survey  questions),  the  number  of  respondents  giving  the  process  of  joining  the  forum  a  moderate 
rating  was  approximately  3.  Three  of  seven  people  on  average  gave  the  process  a  high-moderate 
rating  and  approximately  1  gave  it  a  high  rating.  For  this  reason,  Joining  the  Forum  is  assigned 
an  amber-green  rating. 

Where  respondents  expressed  an  opinion  about  membership  restrictions,  it  was  that  the 
restrictions  on  membership  are  reasonable,  but  may  require  some  modification.  All  respondents 


*  It  is  unknown  how  representative  of  the  S3-XO.net  member  population  this  small  sample  is.  Only  a  small  sample 
was  taken  in  order  to  evaluate  economically  the  feasibility  of  surveying  members  for  their  impressions  of  the  forum. 
However,  an  attempt  was  made  to  make  the  sample  surveyed  as  representative  of  the  target  membership  as  possible 
by  selecting  the  following:  4  current  S3s  or  XOs  who  started  a  conversation  thread  that  contained  multiple  posts  (1 
person  responded),  2  current  S3s  or  XOs  who  started  a  conversation  thread  that  contained  only  the  single  initiating 
post  (1  person  responded),  7  current  S3s  or  XOs  who  had  not  made  a  conversation  post  but  had  visited  the  site 
within  a  month  of  the  survey  (4  people  responded),  6  current  S3s  or  XOs  who  had  not  made  a  conversation  post  but 
had  become  a  new  member  and  visited  the  site  within  a  month  of  the  survey  (1  person  responded).  Nearly  all  of  the 
members  who  belonged  to  these  categories  were  administered  a  survey.  This  sample  is  the  same  survey  sample  used 
throughout  this  report.  For  the  purpose  of  formally  evaluating  an  APF,  a  much  larger  sample  should  be  surveyed.  A 
larger  sample  will  also  permit  an  analysis  of  the  types  of  members  whose  needs  are  being  met/not  met  by  the  forum. 
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agreed  that  the  application  form  conformed  to  their  expectations  of  an  online  form.  In  general, 
respondents  indicated  that  the  time  it  took  for  membership  to  be  granted  was  24  hours  or  less. 
The  majority  of  respondents  felt  that  the  welcome  letter  was  a  nice  gesture. 

Tone  in  the  Forum 

New  members  who  join  the  forum  are  greeted  with  an  auto-generated  welcome  letter 
addressed  from  the  Chief  Editor.  The  letter  is  addressed  to  “Fellow  Professionals”  and  indicates 
how  the  new  member  can  contribute  to  the  betterment  of  “hundreds  of  units.”  The  letter  also 
indicates  that  the  Chief  Editor  is  there  for  the  new  member  if  he  has  an  idea  or  feedback,  and  will 
do  all  he  can  to  help.  Although  the  welcome  letter  is  important  for  building  a  sense  of 
community  in  the  forum,  there  may  be  some  mismatch  between  the  tone  of  the  welcome  letter 
and  the  tone  that  may  be  expected  by  the  new  member.  That  is,  our  interviews  with  potential 
members  indicated  that  they  would  first  join  a  community  to  meet  a  need  and  only  later,  if  the 
need  were  consistently  satisfied  (and  they  had  time),  would  they  feel  an  obligation  to  contribute. 
In  addition,  Gerber  (2003)  noted  that  three  key  determinants  of  whether  members  of  the  target 
audience  would  use  S3-XO.net  were  the  utility  of  information,  ease  of  use,  and  ability  to 
network.  It  may  be  more  effective  for  the  welcome  letter  to  place  greater  emphasis  on  how  the 
forum  will  support  the  member  than  how  the  member  can  support  the  forum.  It  would  also  be 
preferable  if  the  welcome  letter  were  addressed  to  the  new  member  individually  or  to  “Fellow 
Professional”  in  the  singular.  This  may  make  the  form  letter  approach  seem  slightly  more 
personalized.  Finally,  in  the  very  first  paragraph  of  the  welcome  letter  it  is  indicated  that  the 
password  provided  during  registration  is  incorrect  and  should  be  ignored.  This  does  not  set  the 
tone  that  the  forum  support  staff  is  on  top  of  things  technologically,  and  may  make  the  user 
skeptical  that  the  online  meeting  space  will  be  usable.  Of  the  19  S3-XO.net  members  surveyed, 
the  majority  of  respondents  indicated  that  they  felt  the  welcome  letter  was  a  nice  gesture,  but 
none  indicated  that  the  welcome  letter  made  them  feel  like  a  valued  new  member  of  the  forum. 

At  the  top  of  the  S3-XO.net  homepage  the  overall  function  of  the  forum  “Your 
connection  for  building  better  units”  is  clearly  stated  (e.g.,  compare  to  CC.mil:  “Building 
combat-ready  teams!”).  The  main  page  also  presents  a  short  description  of  the  forum’s  purpose, 
stated  as  follows:  “A  professional  forum  where  professionals  can  come  together  to  discuss 
current  issues.”  This  statement  sets  up  well  the  expectation  that  forum  members  are  expected  to 
maintain  a  positive,  professional  bearing.  However,  although  this  description  is  accurate,  it  is 
also  somewhat  vague.  Moreover,  as  our  interviews  indicated,  S3s/XOs  initially  entering  the  site 
are  not  particularly  interested  in  having  a  conversation,  but  would  like  to  find  time-saving 
information  quickly.  The  introductory  text  on  the  main  page,  containing  update  information,  may 
be  a  suitable  place  to  set  a  positive  tone  in  a  manner  that  is  more  engaging  or  directly  relevant  to 
the  typical  user.  For  example,  this  text  includes  a  description  of  what  the  forum  has  to  offer  its 
members  as  well  as  a  statement  describing  the  security  restrictions  on  what  information  can  be 
shared  in  the  forum.  There  is  no  Frequently  Asked  Questions  section  available  in  the  forum,  nor 
is  there  a  user  comments  area,  an  area  for  press  releases  or  other  information,  or  a  brief  tutorial 
or  introduction  to  the  online  meeting  space. 

There  are  two  additional  characteristics  of  the  software  used  to  build  the  online  meeting 
space  that  may  generate  a  tone  not  intended  by  the  forum  support  personal,  namely  that  members 
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are  not  truly  free  to  use  their  judgment  to  make  posts.  The  first  characteristie  is  that  diseussion 
posts  made  by  a  member  who  has  been  purged  from  the  forum  remain  in  the  forum,  but  the 
author  designation  changes  from  the  author’s  name  to  “purged  user.”  Members  are  purged  from 
the  forum  for  reasons  other  than  malfeasance  (e.g.,  the  forum  grows  two  large,  and  members  of 
subgroups  split  off  into  different  forums),  but  this  is  unknown  to  the  current  members.  In  one 
case  in  S3-XO.net,  a  purged  user  happened  to  have  authored  a  discussion  post  that  is  critical  of 
the  logistical  information  management  system  used  in  Iraq.  Because  this  is  the  only  discussion 
post  in  the  forum  that  takes  a  noticeably  critical  tone,  readers  of  this  post  may  assume  that  the 
user  was  purged  for  being  critical  of  the  Army.  The  second  characteristic  is  that  discussion  posts 
that  have  been  modified  by  forum  support  personnel  (e.g.,  to  correct  spelling  or  make  other 
minor  changes)  are  appended  with  a  statement  that  they  have  been  modified  and  who  did  the 
modifying,  but  no  indication  of  why  modifications  were  made.  This  may  create  the  impression 
that  members  are  not  in  control  of  how  their  posts  appear  to  others.  Efforts  to  set  the  tone  in  S3- 
XO.net  is  here  rated  a  combined  red/amber  for  the  above  reasons. 

Conversation  Support 

S3-XO.net  is  built  on  Tomoye  Simplify,  software  specifically  designed  to  support  online 
social  interaction.  Tomoye  allows  members  to  build  discussions  around  “knowledge  objects” 
posted  to  the  forum  (e.g.,  documents,  video  clips,  presentations,  etc.)  and/or  to  generate  new 
conversations  on  related  topics  that  are  not  specifically  associated  with  an  upload  to  the  site. 
Powering  discussion  in  this  way,  the  use  of  Tomoye  solves  the  problem  of  where  to  “locate” 
discussion  in  the  forum  because  discussion  is  enabled  everywhere  (see  Schweitzer,  2003). 
Because  of  its  flexibility  and  ease  of  use,  Tomoye  is  also  the  preferred  software  solution  for 
supporting  discussion  forums  within  the  Army  and  in  other  branches  of  the  service  (e.g..  Navy; 
Kendall  &  McHale,  2003)  and  is  recommended  for  the  commercial  sector  (Schweitzer,  2003). 

Discussion  thread  titles  in  S3-XO.net  are  authored  by  the  initiator  of  the  conversation,  and 
generally  reflect  the  conversation  topic  (e.g.,  “Request  for  a  division  TACSOP  [tactical  standing 
operating  procedure]”  and  “Single  COA  [course  of  action]  or  Multiple?”).  However,  one  or  two 
threads  have  titles  that  are  not  related  at  all  and  suggest  that  the  author  was  uncertain  how  to 
make  a  discussion  post.  For  example  “how  can  I  ask  for  information  or  start  a  discussion”  is  the 
title  for  a  conversation  initiated  by  a  request  for  information  about  improvised  explosive  devices. 
One  or  two  initial  conversation  posts  are  duplicated  in  the  forum,  suggesting  that  the  first  time 
they  made  their  posts  these  users  were  uncertain  whether  they  had  successfully  uploaded  them  to 
the  forum.  Despite  the  occasional  glitch,  the  conversation  capability  of  Tomoye  appears  to  work 
well,  and  includes  helpful  information  for  evaluating  each  post,  such  as  its  author  and  the  date 
and  time  of  the  post.  Currently,  there  is  no  capability  for  members  to  rate  the  utility  of  a 
conversation  thread  or  individual  discussion  post.  Because  the  current  version  of  Tomoye  is  the 
backbone  of  a  highly  active  APF  (CC.mil),  Conversation  Support  was  rated  green. 

Locating  Content 

S3-XO.net  has  three  methods  available  for  locating  content:  a  general  search  capability 
located  at  the  upper  right  of  the  main  page,  a  knowledge  search  capability  located  in  the  lower 
left,  and  a  conversation  search  capability  also  located  in  the  lower  left  of  the  main  page.  The 
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general  search  capability  searches  the  entire  forum,  including  conversations,  documents,  bios, 
and  even  topics,  using  keywords  entered  by  the  user.  The  knowledge  search  allows  users  to 
specify  using  keywords  for  the  type  of  knowledge  they  would  like  to  find.  Searchable  knowledge 
types  include:  Joint  Planning;  Chief  of  Staff  of  the  Army  Task  Force  Updates;  References;  Field 
Interview;  Tactics,  Techniques  and  Procedures,  Lesson  Learned;  Topic;  Dog  Tag  (i.e.,  member 
bio  sketches);  Institution;  Multimedia  File;  Tool/Document;  Website  Link;  Discussion;  News 
Article;  Event;  Announcement;  and  Community.  Once  a  knowledge  type  is  selected,  the  user  can 
indicate  what  fields  he  would  like  displayed  and  how  he  would  like  the  results  arranged  by 
pressing  a  “filter”  button  and  selecting  items  from  a  pull-down  menu®.  Unfortunately,  the 
knowledge  search  capability  does  not  have  a  “find”  or  “go”  button,  making  it  potentially  robust 
but  currently  not  functional  because  there  is  no  way  to  initiate  the  search.  The  conversation 
search  allows  the  user  to  search  for  keywords  present  in  discussion  threads.  All  of  these  search 
functions  allow  the  user  to  sort  search  results  in  ascending  or  descending  order,  but  it  is  unclear 
what  is  ascending  or  descending  (e.g.,  number  of  posts  or  number  of  views  or  some  other  metric, 
such  as  date  posted). 

Each  topic  and  sub-topic  in  the  forum  has  a  link  that  allows  members  to  view  the 
conversation  and  knowledge  shared  on  that  topic.  Topics  with  recent  posts  are  flagged  with  a 
flashing  callout  to  indicate  that  a  new  conversation  has  taken  place.  In  addition,  a  number 
featured  next  to  the  topic  or  sub-topic  name  indicates  the  activity  associated  with  the  topic. 
Activity  associated  with  individual  posts  to  the  forum  (i.e.,  the  number  of  times  a  post  has  been 
viewed)  is  indicated  by  numbers  that  appear  near  the  post’s  title.  However,  the  metrics  used  to 
track  the  number  of  views  is  currently  inaccurate,  the  topic  activity  metric  is  not  well  understood, 
and  the  algorithm  behind  the  flags  indicating  recent  posts  is  not  reliable.  There  is  a  “Rate  this 
Page”  functionality,  which  allows  the  user  to  rate  a  particular  page  on  a  scale  of  1  (worst)  to  5 
(best).  This  rating  functionality  currently  appears  to  exist  only  on  the  main  page,  however,  and 
also  does  not  apply  to  individual  content  contributions  on  the  page. 

Members  of  S3-XO.net  receive  an  announcement  via  email  that  there  is  a  monthly 
newsletter  posted  in  the  forum.  The  newsletter  is  accessed  by  clicking  a  hotlink  on  the  main  page 
called  “Net  Call  (Month)  (Year).”  There  currently  is  no  content  analysis  or  synthesis  conducted 
in  the  forum.  However,  the  current  and  future  S3s/XOs  we  interviewed  indicated  that  content 
that  is  analyzed  and  synthesized  will  be  of  maximal  use  to  them.  Links  between  content  within 
the  forum  are  possible,  and  can  be  made  either  by  the  facilitators  or  by  forum  members.  Links  to 
content  outside  the  forum  [e.g.,  at  the  Center  for  Army  Lessons  Learned  (CALL)]  may  also  be 
made  or  content  may  be  brought  in  and  made  downloadable  (e.g.,  field  manuals).  The  forum 
provides  links  to  CALL,  BCKS,  AKO,  the  Joint  Electronic  Library,  and  the  Army  Library 
Program. 

Four  survey  questions  (complete  survey  is  located  in  Appendix  D)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  finding  content  in  the  forum.  These  questions  addressed  the  ease  of 
navigating  content  topics,  using  the  search  capability,  finding  new  content  additions,  and 
understanding  the  arrangement  of  content.  On  average,  approximately  3  respondents  gave 


^  There  are  different  pull-down  menu  options,  depending  on  what  knowledge  type  is  selected.  Pressing  the  “filter” 
button  links  the  knowledge  type  selected  by  the  user  to  its  associated  data  fields. 
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finding  content  in  the  forum  a  high-moderate  rating.  Two  people  on  average  gave  finding  content 
a  high  rating,  2  gave  it  a  moderate  rating,  and  approximately  1  gave  it  a  low-moderate  rating. 

In  general,  respondents  indicated  that  navigating  content  topics  was  easy  or  fairly  easy. 
Most  respondents  indicated  that  the  search  capability  worked  fairly  well  with  only  occasional 
problems  or  that  they  had  not  noticed  anything  special  about  the  search  capability  one  way  or  the 
other.  Respondents  generally  indicated  that  they  had  no  trouble  finding  the  new  content  additions 
to  the  forum.  Respondents  also  agreed  that  the  way  content  is  arranged  in  the  forum  generally 
makes  sense  or  that  they  had  not  noticed  anything  particular  about  how  the  content  was  arranged, 
good  or  bad.  For  all  of  the  above  reasons.  Locating  Content  was  assigned  an  amber  rating. 

Locating  People 

Through  our  interviews,  we  discovered  that  by  the  time  officers  are  ready  to  assume  the 
S3/XO  duty  position,  they  have  already  established  many  relationships  within  the  organization, 
and  have  made  decisions  about  who  to  trust.  They  must  be  able  to  locate  people  in  the  forum 
they  already  know  and  respect  in  order  to  feel  that  the  forum  can  be  of  use  to  them.  The  ability  to 
establish  the  credibility  of  strangers  is  critical  in  their  determination  of  the  validity  (i.e.,  veracity 
and  relevance)  of  posted  content.  The  future  S3s/XOs  we  interviewed  also  indicated  that  it  would 
be  highly  desirable  to  be  able  to  locate  the  current  S3s/XOs  in  the  units  they  are  preparing  to 
join.  They  are  unable  to  locate  this  information  easily  through  AKO  or  through  installation 
homepages. 

S3-XO.net  has  a  member  directory  that  allows  members  to  alphabetically  search  for 
someone  they  think  may  be  a  member  of  the  forum  or  to  search  for  members  depending  on 
certain  biographical  criteria  (e.g.,  duty  status,  assignment  history,  branch,  etc.).  Biographical 
sketches  must  include  full  name  and  email  address.  Sketches  may  also  include  home  and  duty 
phone,  duty  status,  branch,  current  duty  position,  current  duty  station,  deployments,  S3/XO 
experience,  topic  areas  of  expertise,  and  links  to  the  member’s  most  recent  posts  (knowledge  or 
conversation).  Clicking  on  the  name  of  a  contributor  to  the  forum  allows  members  to  access  that 
contributor’s  biographical  data.  The  main  page  of  S3-XO.net  features  a  list  of  the  members  who 
are  currently  logged  in  to  the  forum.  These  names  can  also  be  clicked  on  to  retrieve  biographical 
data.  There  currently  is  not  a  “featured  person”  area  in  the  forum.  However,  S3-XO.net  does 
enable  members  to  indicate  that  they  are  an  “interested  party”  in  a  particular  topic  (e.g.,  fire 
support,  leader  development,  etc.).  Experts  in  the  topics  for  which  there  is  an  interested  party 
may  then  direct  their  assistance  to  these  members. 

Locating  People  was  assigned  a  green  status,  though  in  the  specific  case  of  S3-XO.net,  it 
would  be  extremely  beneficial  to  feature  an  additional  directory  that  provides  the  contact 
information  of  members  and  non-members  holding  BN/BDE  S3/XO  duty  positions  in  units 
Army-wide. 
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Structure  and  Characteristics  -  Recommendations 


The  Plan 

Several  sources  have  suggested  that  one  can  build  a  forum  but  that  a  separate,  even  more 
challenging  effort  is  to  get  people  to  come  to  that  forum  (e.g.,  Dixon  et  ah,  2005;  Kendall  & 
McHale,  2003;  Wenger  et  ah,  2002).  Potential  users  must  feel  comfortable  with  the  forum 
culture  and  confident  that  the  forum  will  meet  their  needs.  An  in-depth  understanding  of  the 
target  audience  is  required  to  bring  the  forum  to  the  attention  of  potential  members  and  to  ensure 
that  initial  activity  in  the  forum  is  directed  to  member  sensibilities  and  needs  (Gerber,  2003; 
Stuckey  &  Smith,  2004;  Wenger  et  ah,  2002). 

The  overall  rating  of  S3-XO.net' s  plan  is  amber,  with  highest  ratings  for  functional  and 
technical  specification  and  lowest  ratings  for  identification  and  understanding  of  the  target 
audience.  The  strengths  and  weaknesses  of  the  current  plan  suggest  that  emphasis  has  been 
placed  on  the  implementation  of  the  forum  at  the  cost  of  the  purpose  and  goals  of  the  forum. 
Particularly  because  S3-XO.net' s  founding  members  were  not  current  S3s  and  XOs  and  because 
the  rapid  institutionalization  of  the  forum  involves  civilian  contractors  in  the  facilitation  process, 
it  is  essential  that  the  S3-XO.net  support  staff  aggressively  pursue  an  understanding  of  the  target 
audience’s  needs.  This  could  include  visits  to  U.S.  Army  Forces  Command  (FORSCOM) 
installations  to  talk  with  S3s/XOs  and  their  commanders,  discussions  with  battlestaff 
observer/controllers  at  the  combat  training  centers,  discussions  with  officers  in  the  pre-command 
courses  who  have  recently  left  S3/XO  positions,  and  interviews  with  S3s  and  XOs  recently 
returned  from  deployment.  Although  future  S3s/XOs  make  up  part  of  both  the  group  of  founding 
officers  and  the  target  audience,  these  officers  do  not  always  know  what  their  knowledge  needs 
are  and  will  benefit  from  the  discussion  of  current  S3s/XOs  as  they  hash  out  their  challenges  in 
the  forum. 

In  the  end,  the  forum  leadership  and  the  other  personnel  supporting  the  forum  should  be 
able  to  articulate  not  only  how  they  think  participation  in  S3-XO.net  will  enhance  unit 
effectiveness,  but  also  where  specifically  they  expect  to  see  impact.  Although  a  complete  front- 
end  analysis  of  the  S3-XO.net  target  audience  is  beyond  the  scope  of  the  present  project,  later 
sections  of  this  report  will  hopefully  begin  to  address  this  difficult  but  critical  issue. 

Personnel 

Even  with  a  solid  understanding  of  the  target  audience,  a  great  deal  of  activity  on  the  part 
of  a  core  group,  facilitators,  and  leadership  is  required  to  generate  self-sustaining  APF  activity 
(Dixon  et  ah,  2005;  Kendall  &  McHale,  2003).  The  S3-XO.net  leadership  and  support  personnel 
are  extremely  competent  and  passionate  about  officer  professional  development  but 
unfortunately  must  juggle  the  needs  of  the  forum  with  many  other  job  duties.  The  forum  requires 
several  personnel  dedicated  full-time  to  accomplish  its  facilitation  and  marketing  needs.  These 
individuals  ideally  would  be  Army  officers  who  have  just  completed  S3/XO  duty  positions,  who 
have  elected  the  role  of  shepherding  the  forum,  and  who  are  reflective  about  their  careers  and 
their  profession.  If  coupled  with  the  ideal  support  personnel  described  above,  additional  full-time 
civilian  contractors  would  also  be  a  valuable  aid  in  facilitation.  However,  clarification  as  to  who  , 
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has  primary  responsibility  for  locating  individuals,  connecting  members,  and  placing  content  in 
the  forum  must  be  clearly  determined  before  additional  personnel  are  dedicated  to  the  forum. 

Functional  Requirements 

The  overall  status  of  S3-XO.net' s  functional  requirements  is  amber/green,  reflecting,  as 
did  the  technical  and  functional  specification  of  the  forum  plan,  the  strength  behind  S3-XO.  net's 
online  meeting  space.  Tomoye  Simplify  supports  robust  social  interaction,  and  pending  updates 
to  the  software  will  enhance  its  potentially  robust  search  capabilities.  Where  S3-XO.net' s 
functional  requirements  need  improvement— Finding  the  Forum  and  Tone  in  the  Forum — ^recent 
initiatives  by  the  forum  leadership  and  support  staff  suggest  a  promising  future  direction  in 
recruiting  and  retaining  the  interest  of  new  members. 

S3-XO.net  Social  and  Intellectual  Capital 
Connections 


Table  11.  S3-XO.net  Connection  Characteristics  Status 


Connection  Characteristics 

Status 

Amber 

Green 

Membership  Composition 

Participating  Members 

Identifying  Experts 

Facilitation  of  Connections 

Connection  Quality 

Overall  Status 

Membership  Composition 

S3-XO.net  membership  data  were  downloaded  from  the  forum  on  8  April  2005.  To 
prepare  these  data  for  analysis,  duplicate  accounts  (33),  guest  accounts  (1),  invalid  accounts  (3), 
and  purged  user  accounts  (113)  were  deleted.  In  addition,  incorrect  data  verifiable  through  AKO, 
such  as  title  and  duty  status,  were  corrected'^.  Of  the  221 5  members  retained  for  analysis,  697 
accounts  were  missing  significant  portions  of  biographical  data  (i.e.,  all  optional  fields  were 
blank).  Because  the  S3/XO  experience  of  the  members  laeking  biographical  data  could  not  be 
determined  using  AKO,  these  697  members  are  not  included  in  the  below  analysis  of 
membership  composition.  This  creates  a  sample  size  of  1518  for  S3-XO.net  membership, 
approximately  69%  of  the  total  population  of  S3-XO.net  members  as  of  8  April  2005. 

Among  the  1518  members  in  the  S3-XO.net  membership  sample,  221  reported  being 
eurrent  S3s/XOs  and  285  reported  being  future  S3s/XOs.  Therefore,  33%  of  the  total 


This  was  done  infrequently  on  a  case-by-case  basis  where  inconsistencies  in  the  biographical  data  suggested  an 
error.  For  example,  a  DA  civilian  or  retired  officer  may  list  their  status  as  “active  duty”  instead  of  “other”  and 
“retired,”  respectively.  Some  reserve  officers  listed  themselves  as  “Mr.”  instead  of  using  their  military  title. 
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membership  sample  is  a  member  of  the  target  audience’  ’ .  Forty-five  percent  of  the  target 
membership  is  a  current  S3/XO  and  55%  is  a  future  S3/XO.  Assuming  the  697  accounts  lacking 
biographical  data  are  representative  of  the  1518  accounts  including  biographical  data,  we  could 
conclude  that  as  of  8  April  2005  there  are  73 1  target  members  in  the  total  membership 
population,  with  329  current  S3s/XOs  and  402  future  S3s/XOs.  According  to  Gerber  (2003),  the 
approximate  size  of  the  target  audience  for  S3-XO.net  is  23,545.  With  731  current  or  future 
S3s/XOs,  S3-XO.net  would  have  approximately  3%  of  the  target  audience  in  its  membership. 

It  is  not  certain  that  those  members  with  significant  portions  of  biographical  data  missing 
are  representative  of  the  sample  of  members  whose  biographical  data  are  filled  out.  It  may  be  the 
case  that  biographical  data  for  some  members  were  missing  intentionally,  for  example  for  very 
high-ranking  officers  or  non-commissioned  officers.  Other  members  who  did  not  provide 
biographical  data  may  not  make  have  taken  the  time  to  do  so  because  they  do  not  belong  to  the 
target  membership  and  therefore  do  not  see  themselves  as  active  participants  in  the  forum.  In  any 
event,  the  difference  between  73 1  members  of  the  target  audience  (when  using  the  entire 
membership  population)  and  506  members  (when  using  the  membership  sample)  is  trivial 
relative  the  size  of  the  estimated  target  audience— a  difference  of  one  percentage  point. 

Because  S3-XO.net  members  represent  only  2-3%  of  the  target  audience,  and  only  33% 
of  the  of  forum  membership  is  a  member  of  the  target  audience,  but  there  is  a  nearly  50-50 
balance  among  current  and  future  S3s/XOs,  the  assigned  status  for  Membership  Composition  is 
red/amber.  Note  that  since  8  April  2005,  there  have  been,  on  average,  1 8  new  people  added  to 
the  S3-XO.net  membership  each  week.  Between  8  April  and  23  June  2005,  202  new  members 
have  been  added,  45%  of  which  are  target  members.  Although  this  rate  of  new  membership  is 
somewhat  sluggish,  it  may  reflect  a  promising  trend  of  increased  percentage  of  target  members 
in  the  overall  membership.  The  balance  among  current  and  future  S3s/XOs  in  the  recently  added 
target  membership  also  represents  promising  movement  in  a  positive  direction  (54-46  compared 
to  43-57). 

Participating  Members 

Conversation.  To  examine  conversation  composition,  the  principal  investigator  reviewed 
the  344  discussion  posts  present  under  “Conversations”  headings  in  S3-XO.net  as  of  12  May 
2005.  Forum  members  who  did  not  indicate  their  S3/XO  experience  were  removed  from  the 
analysis,  reducing  the  number  of  posts  to  327.  As  shown  in  the  table  below,  34%  of  the 
conversation  posts  on  S3-XO.net  were  made  by  target  members.  This  percentage  of  contributions 
may  be  expected  given  that  an  estimated  33%  of  the  total  membership  is  a  target  member. 
Although  the  criteria  for  evaluating  target  member  participation  in  conversation  [“Green” 
(85%+)]  are  arbitrary  in  the  sense  that  no  particular  percentage  of  target  member  participation  in 
conversation  has  ever  been  linked  to  community  health  or  impact,  common  sense  suggests  that  a 
majority  of  the  members  participating  in  the  conversation  should  be  from  the  target  membership 
in  order  for  conversation  to  be  relevant  and  to  have  a  potential  impact  in  the  target  community 


"  See  Section  I.  The  remaining  67%  of  the  sample  membership  is  composed  of  175  former  S3s/XOs,  834  members 
with  “N/A”  indicated  for  their  S3/XO  experience,  and  3  missing  entries.  Eighty-seven  percent  of  the  834  members 
with  “N/A”  indicated  for  their  S3/XO  experience  were  CGSC  students,  evidently  not  on  track  to  become  an  S3  or 
XO.  The  remaining  13%  had  various  duty  positions,  including  either  military  or  civilian  positions. 
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(see  also  Gerber,  2003;  Stuckey  &  Smith,  2004).  Thirty- four  percent  falls  short  of  the  most 
liberal  definition  of  majority  (51%  or  more). 

If  the  founding  officers  are  considered  target  members,  the  percentage  of  conversation 
posts  made  by  target  members  rises  to  just  below  46%  (this  percentage  does  not  include  the 
contributions  of  one  founding  officer  with  multiple  postings  who  is  a  past  S3/XO).  Of  the 
original  34%  of  conversation  posts  made  by  target  members,  24%  of  these  posts  was  made  by 
current  S3s  or  XOs  (compare  to  the  fact  that  an  estimated  45%  of  the  target  members  are  current 
S3s  or  XOs).  If  the  founding  officers  are  included,  approximately  17%  of  the  conversation  posts 
were  made  by  current  S3s  or  XOs. 


Table  12.  Conversation  Posts  by  S3-XO.net  Member  Role 


Member  Role 

#  of  Conversation  Posts 

Percentage  of  Total  Conversation  Posts 

Target  Member 

111 

34% 

Founding  Officer 

50 

15% 

S3-XO.net  Staff 

62 

19% 

Former  S3/XO 

53 

16% 

N/A 

51 

16% 

TOTAL 

327 

100% 

Note  that  the  discussion  in  the  forum  may  not  represent  the  total  amount  of  discussion 
that  has  occurred  due  to  the  forum.  Forum  members  may  initiate  or  continue  discussion  outside 
of  the  forum  through  email,  telephone,  or  face-to-face  contact.  Contact  outside  of  the  forum  may 
increase  the  impact  of  forum  activity  on  officer  development.  Moreover,  it  is  more  likely  that 
target  members  will  initiate  and  maintain  discussion  outside  of  the  forum  than  non-target 
members.  Discussion  and  knowledge  sharing  inside  the  forum,  however,  must  be  relevant  to 
target  member  needs  in  order  to  stimulate  discussion  and  knowledge  sharing  outside  the  forum. 
Relevance  may  be  at  risk  if  relatively  few  target  members  are  participating. 

Knowledge.  To  examine  knowledge  contributions,  a  sample  of  161  knowledge  objects 
posted  to  the  site  as  of  12  May  2005  were  reviewed.  This  sample  size  reflects  slightly  more  than 
half  of  the  total  number  of  knowledge  posts'^.  Four  posts  were  removed  from  the  below  analyses 
because  the  S3/XO  experience  of  the  member  having  made  the  post  could  not  be  determined. 
Knowledge  post  types  reviewed  included  tools/documents,  website  links,  announcements,  books, 
multimedia  files,  and  ideas/tactics,  techniques,  and  procedures/lessons  learned.  These  type 
categories  mirror  those  used  in  S3-XO.net  to  organize  information.  As  shown  in  the  table  below, 
17%  of  the  knowledge  posts  on  S3-XO.net  were  made  by  target  members.  If  the  founding 
officers  are  considered  target  members,  this  percentage  increases  to  a  majority  contribution  of 
60%.  Of  the  original  17%  of  knowledge  posts  contributed  by  target  members,  26%  was  made  by 
current  S3s/XOs  (compare  to  the  fact  that  an  estimated  45%  of  the  target  members  are  current 
S3s  or  XOs).  When  the  founding  officers  are  considered  target  members,  this  percentage 


Our  estimate  of  approximately  283  knowledge  posts  differs  significantly  from  the  estimate  of  5300  knowledge 
posts  featured  on  the  S3-XO.net  homepage.  Our  estimate  does  not  include  conversation  posts  (approximately  341), 
member  “dog  tags”  or  bio  sketches  (approximately  2215),  documents  on  S3-XO.net  itself,  located  in  the  archives 
(approximately  24),  topics  (unknown  number),  institution  descriptions  (unknown  number),  or  communities 
(unknown  number). 
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decreases  to  8%  (this  percentage  does  not  include  the  contribution  of  one  founding  officer  who  is 
a  past  S3/XO). 

In  the  case  of  both  conversation  posts  and  knowledge  posts,  current  S3s  and  XOs  were 
contributing  at  a  lower  rate  than  they  are  represented  in  the  target  membership.  However,  current 
S3s  and  XOs  do  not  appear  to  be  visiting  the  site  at  a  lower  rate  than  they  are  represented  in  the 
target  membership.  In  12  samples  of  S3-XO.net’ s  “members  eurrently  online”  feature  between  18 
May  and  10  June  2005,  the  proportion  of  current  S3s/XOs  online  (relative  to  future  S3s/XOs) 
roughly  matched  the  proportion  of  current  S3s/XOs  (relative  to  future  S3s/XOs)  in  the  target 
membership.  Whereas  current  S3s/XOs  make  up  45%  of  the  target  membership  (as  of  8  April), 
the  average  percentage  of  the  target  membership  online  (between  18  May  and  10  June)  that  was 
a  current  S3  or  XO  was  49%.  It  is  possible  to  conclude  from  this  brief  sampling  of  members 
online  that  current  S3s/XOs  are  less  likely  than  future  S3s/XOs  to  post  when  they  are  online,  not 
that  they  are  online  less  frequently.  In  any  case,  because  target  members  are  making  a  minority 
of  the  contributions  both  in  terms  of  conversation  and  knowledge  unless  the  founding  officers  are 
considered,  and  the  founding  officers  no  longer  actively  participate  in  the  forum.  Participating 
Members  was  assigned  a  red  status  rating. 


Table  13.  Knowledge  Posts  by  S3-XO.net  Member  Role 


Member  Role 

#  of  Knowledge  Posts 

Percentage  of  Total  Knowledge  Posts 

Target  Member 

27 

17% 

Founding  Officer 

67 

43% 

S3-XO.net  Staff 

31 

20% 

Former  S3/XO 

17 

11% 

N/A 

15 

10% 

TOTAL 

157 

100% 

Identifying  Experts 

Sixty-nine  percent  ( 1 5 1 8/22 1 5)  of  forum  members  provided  complete  or  nearly  complete 
biographical  data,  which  other  members  could  use  to  locate  experts.  However,  only  17% 
provided  information  regarding  areas  of  expertise.  Of  the  current  S3s/XOs,  44%  provided 
expertise  data;  future  S3s/XOs,  46%;  and  previous  S3s/XOs,  50%.  In  comparison,  only  4%  of 
S3-XO.net  members  who  selected  “N/A”  for  S3/XO  experience  provided  information  regarding 
areas  of  expertise.  Given  that  the  large  majority  (87%)  of  the  members  who  selected  “N/A”  for 
S3/XO  experience  did  not  expect  to  become  an  S3  or  XO  and  may  not  see  themselves  as 
contributors  to  the  site,  it  is  reasonable  to  expect  low  level  of  expertise  data  from  these  people.  In 
contrast,  the  key  members  of  the  forum  (target  members  and  experts)  provided  much  higher  and 
roughly  equivalent  levels  of  expertise  data. 

Three  survey  questions  (see  Appendix  D  for  the  complete  survey)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  finding  experts  in  the  forum.  These  questions  addressed  the  ease  of 
finding  experts  in  the  forum,  the  ease  of  finding  experts  external  to  the  forum,  and  the  confidence 
respondents  had  in  expert  members’  willingness  to  help  them.  On  average,  3  of  the  7  respondents 
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gave  finding  experts  a  high-moderate  rating.  Approximately  2  people,  on  average,  gave  finding 
experts  a  moderate  rating,  and  approximately  2  gave  it  a  high  rating. 

Interestingly,  for  finding  experts  either  in  the  forum  or  because  of  the  forum  (external 
experts),  the  modal  response  was  that  respondents  had  not  given  thought  to  using  the  forum  to 
locate  experts.  Of  those  who  did  consider  using  the  forum  for  finding  experts,  roughly  equal 
proportions  indicated  that  they  were  either  confident  or  certain  that  the  forum  is  a  useful  resource 
for  locating  experts.  Respondents  generally  agreed  that  they  were  somewhat  confident  a  forum 
member  would  help  them  if  asked,  with  some  respondents  indicating  that  they  were  completely 
confident.  For  these  reasons,  finding  experts  was  assigned  an  amber  rating. 

Facilitation  of  Connections 

As  mentioned  previously,  S3-XO.net  enables  members  to  indicate  that  they  are  an 
“interested  party”  in  a  particular  topic,  and  experts  in  these  topics  may  then  direct  their  assistance 
to  these  members.  However,  it  may  be  unlikely  that  an  expert  will  contact  an  interested  party  for 
an  open-ended  discussion  if  a  specific  question  has  not  been  asked,  so  it  is  unclear  how  effective 
this  feature  is  in  facilitating  connections.  The  sole  person  assigned  to  full-time  support  of  S3- 
XO.net  reported  not  having  specific  connections  with  experts  in  any  of  the  content  areas  in  the 
forum,  though  he  did  indicate  that  he  was  able  to  call  on  personnel  stationed  at  Fort  Leavenworth 
to  help  with  answering  some  member  questions. 

Two  survey  questions  (see  Appendix  D  for  the  complete  survey)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  posting  questions  in  the  forum.  These  questions  addressed  how 
question  posts  were  acknowledged  and  confidence  in  the  facilitators’  ability  to  ensure  that 
questions  were  adequately  answered.  Of  the  seven  respondents  to  the  survey,  only  four  had 
posted  a  question.  On  average,  five  respondents  gave  posting  questions  a  moderate  rating. 
Approximately  one  person,  on  average,  gave  posting  questions  a  high-moderate  or  high  rating. 
One  person  on  average  gave  posting  questions  a  low-moderate  rating. 

Of  the  four  respondents  who  posted  a  question  to  the  forum,  the  majority  could  not 
remember  if  they  had  received  an  acknowledgement  of  their  posting,  but  one  member  recalled 
never  receiving  an  acknowledgement.  The  majority  of  respondents  also  indicated  that  they  had 
not  considered  how  far  they  thought  forum  facilitators  would  go  to  help  them  answer  a  question. 
Of  those  respondents  who  had  given  it  some  consideration,  all  were  confident  that  the  facilitators 
would  make  an  effort  to  help  them  answer  their  question,  with  equal  proportions  indicating  they 
thought  the  facilitators  would  go  “all  out”  versus  “only  so  far.”  For  these  reasons.  Facilitation  of 
Connections  was  assigned  a  red  rating. 

Connection  Quality 

Three  survey  questions  (see  Appendix  D)  asked  S3-XO.net  members  (see  footnote  #8  in 
Joining  the  Forum  for  a  description  of  the  survey  sample)  to  indicate  their  impressions  of  making 
connections  in  the  forum.  These  questions  addressed  the  ease  of  making  connections  in  the 
forum,  the  utility  of  these  connections,  and  the  likelihood  of  re-using  connections  to  get 
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additional  help.  On  average,  four  people  gave  making  eonnections  a  high-moderate  rating. 
Approximately  two  respondents  on  average  gave  making  connections  a  moderate  rating,  and 
zero  a  high  rating. 

Interestingly,  the  majority  of  respondents  indicated  having  no  opinion  about  how 
effective  it  was  to  seek  help  in  the  forum.  Of  those  who  expressed  an  opinion,  there  was 
agreement  that  help  is  generally  received  by  those  who  request  it.  The  majority  of  those  surveyed 
also  indicated  that  experts  in  the  forum  seemed  to  be  a  valuable  resource  but  that  they  had  not 
personally  benefited  yet  from  these  experts.  Finally,  the  majority  of  respondents  indicated  that, 
depending  on  the  person,  they  would  feel  comfortable  asking  for  help  multiple  times  from  an 
expert  whom  they  had  met  in  forum.  Because  the  members  we  surveyed  seemed  positive  about 
the  quality  of  connections  in  the  forum,  but  had  not  been  actively  seeking  or  had  benefited  from 
a  connection.  Connection  Quality  was  assigned  an  amber  rating. 

Context 


Table  14.  S3-XO.net  Context  Characteristics  Status 


Context  Characteristics 

Status 

Red 

Amber 

Green 

Shared  Narratives 

Shared  Codes  and  Language 

Overall  Status 

Shared  Narratives 

S3-XO.net  provides  opportunities  to  participate  in  shared  experiences  via  book  reviews. 
“Book  Reviews”  is  a  feature  inside  the  “Professional  Development”  area  on  the  site.  One  book, 
American  Soldier,  had  been  reviewed  in  a  discussion  thread  featuring  six  conversation  posts 
among  two  future  S3s/XOs  and  three  forum  members  who  selected  “N/A”  for  their  experience  as 
an  S3/XO.  Upon  examination  of  the  discussion  thread  around  the  book,  however,  it  is  not  clear 
whether  it  should  be  classified  as  a  shared  experience.  That  is,  after  the  second  posting  in  the 
thread,  the  conversation  moved  away  from  the  book  to  other  topics,  and  it  is  not  clear  from  their 
contributions  that  members  joining  the  conversation  late  in  the  process  actually  read  the  book. 
The  book  evidently  was  as  effective  in  generating  conversation  as  other  knowledge  resources 
posted  to  the  site,  but  the  conversation  may  not  have  come  out  of  a  shared  experience  per  se. 

There  were  no  other  formally  designated  opportunities  in  or  through  S3-XO.net  to 
participate  in  shared  experiences.  For  example,  the  electronic  calendar  listing  community  events 
did  not  have  any  events  listed  and  there  were  no  other  types  of  collaborative  exercises. 

Two  survey  questions  (see  Appendix  D)  asked  S3-XO.net  members  (see  footnote  #8  in 
Joining  the  Forum  for  a  description  of  the  survey  sample)  to  indicate  their  impressions  of  shared 
experience  via  the  forum.  These  questions  addressed  the  sense  of  professional  community  gained 
from  the  forum  and  the  sense  of  shared  experience  with  forum  members.  On  average,  five 
respondents  gave  sharing  experience  a  high-moderate  rating.  Approximately  1  person  gave 
sharing  experience  a  moderate  rating,  and  one  gave  it  a  high  rating. 
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The  majority  of  respondents  indicated  that  they  could  envision  a  strong  sense  of 
professional  community  and  shared  experience  developing  in  the  forum,  but  did  not  report  that 
such  a  sense  of  shared  community  or  shared  experience  had  already  developed.  Roughly  equal 
proportions  of  those  surveyed  reported  that  they  had  no  opinion  regarding  a  sense  of  professional 
community  or  shared  experience  or  that  they  had  already  developed  a  sense  of  professional 
community  or  shared  experience.  For  these  reasons,  Shared  Experience  was  assigned  an  amber 
rating. 

Shared  Codes  and  Language 

One  survey  question  (complete  survey  is  located  in  Appendix  D)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  the  nature  of  the  content  in  the  forum.  The  modal  number  of 
respondents  (3)  indicated  that  sometimes  content  is  out  of  their  area  of  interest  (e.g.,  branch),  but 
overall  it  is  not  a  problem.  Roughly  equal  proportions  of  the  remaining  respondents  indicated 
that  they  felt  the  content  generally  spoke  well  to  their  area  of  interest,  that  they  had  not  noticed 
one  way  or  the  other  whether  content  spoke  to  their  area  of  interest,  or  that  they  have  had  to  work 
to  find  content  speaking  their  area  of  interest.  Therefore,  shared  Codes  and  Language  was 
assigned  an  amber  rating. 


Content 


Table  15.  S3-XO.net  Content  Characteristics  Status 


Content  Characteristics 

Status 

Red 

Amber 

Green 

Contributing  Members 

Content  Synthesis 

Content  Organization 

Content  Activity 

Content  Quality 

Professional  Links 

Overall  Status 

Contributing  Members 

As  shown  in  the  table  below,  of  the  157  knowledge  posts  examined,  the  majority  was 
located  in  either  in  the  tools/documents  area  (39%)  in  each  topic  or  situated  within  conversations 
(41%).  The  significant  majority  of  knowledge  posts  found  in  tools/documents  was  made  by  the 
founding  officers  and  facilitators  (82%).  Contributions  of  knowledge  posts  during  conversations 
were  more  evenly  distributed  among  founding  officers  and  facilitators  (34%),  target  members 
(31%),  former  S3s/XOs  (17%),  and  members  who  selected  “N/A”  for  their  S3/XO  experience 
(1 8%).  Target  members  made  nearly  all  of  their  knowledge  posts  (96%)  in  conversation,  rather 
than  in  the  tools/documents  area.  That  the  vast  majority  of  target  member  posts  occurred  in  the 
context  of  a  conversation  suggests  that  target  members  currently  are  more  likely  to  take  the  effort 
to  respond  to  a  specific  request  for  knowledge  sharing,  rather  than  to  respond  to  an  abstract. 
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perceived  need  in  the  larger  community.  This  is  in  contrast  to  the  posting  activity  of  facilitators 
and  the  founding  officers,  who  made  30%  of  their  posts  in  the  context  of  conversations. 

As  shown  previously  in  Table  13,  17%  of  the  S3-XO.net  knowledge  posts  sampled  was 
made  by  target  members,  falling  short  of  a  51%  majority.  If  the  founding  officers  are  considered 
target  members,  this  percentage  increases  to  a  majority  contribution  of  60%.  Of  the  original  17% 
of  knowledge  posts  contributed  by  target  members,  26%  was  made  by  current  S3s/XOs  (compare 
to  the  fact  that  an  estimated  45%  of  the  target  members  are  current  S3s  or  XOs).  When  founding 
officers  are  considered  target  members,  this  percentage  decreases  to  8%.  Because  target 
members  are  making  a  minority  of  knowledge  posts  unless  founding  officers  are  considered,  and 
the  founding  officers  are  no  longer  actively  participating  in  the  forum.  Contributing  Members 
was  assigned  a  red  status  rating. 


Table  16.  Knowledge  Post  Locations 


Location 

#  of  Posts 

Percentage  of  Posts 

Conversations 

65 

41% 

Tools/Documents 

62 

39% 

Website  Links 

22 

14% 

Multimedia  Files 

3 

2% 

Announcements 

3 

2% 

Books 

1 

1% 

Ideas/TTPs/Lessons  Learned 

1 

1% 

Total 

157 

100% 

Content  Synthesis 

Two  of  the  324  conversation  posts  in  the  forum  featured  a  link  to  other  conversations  in 
the  forum.  Both  of  these  posts  were  made  by  the  same  member,  a  future  S3/XO  who  was 
attempting  to  draw  attention  to  the  discussion  thread  he  had  started  in  another  topic  area.  In  one 
case,  the  conversation  in  which  the  link  was  featured  was  related  to  the  target  conversation  of  the 
link.  In  the  other  case  it  was  unrelated.  One  of  the  102  conversations  started  in  the  forum  had 
some  degree  of  summarization  made  by  a  facilitator  who  was  also  a  previous  S3/XO.  Six  of  the 
102  conversations  featured  comments  on  the  discussion  itself  (e.g.,  whether  it  was  on  track), 
which  were  made  by  facilitators,  a  future  S3/XO,  and  members  selecting  “N/A”  for  their  S3/XO 
experience.  Of  the  157  knowledge  posts  reviewed,  two  or  three  were  posted  in  multiple  related 
topics.  One  post,  a  video-captured  interview,  was  broken  into  several  components  and  the 
components  distributed  to  relevant  topics. 

Although  some  attempt  has  been  made  to  cross-reference  posts  in  the  forum,  much  of  the 
related  content  and  conversation  remain  distinct  from  one  another.  In  each  new  topic  page  (e.g., 
Warfighting->  Operations),  frequently  there  are  separate  areas  for  conversation,  tools/documents, 
website  links,  etc.,  but  there  currently  is  no  cross-linking  of  related  posts  in  these  separate  areas. 
For  example,  in  Admin/Log-> Maintenance  Meetings,  there  are  four  separate  but  highly  related 
conversations  about  maintenance  meetings  that  are  not  cross-linked  or  consolidated,  nor  are  they 
cross-linked  with  maintenance  meeting  related  documents  in  the  Tools/Documents  area  on  the 
same  page.  This  is  the  case  across  topic  pages  as  well  as  within  topic  pages.  For  example,  in 
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Warfighting-^Open  Discussion,  there  is  a  discussion  thread  on  how  eombat  training  centers  have 
changed  given  the  new  warfighting  environment.  This  discussion  is  not  referenced,  however,  in 
Training-^ Combat  Training  Centers.  As  another  example,  there  are  several  discussions  and 
knowledge  posts  related  to  the  military  decision-making  process  in  several  different  topic  pages, 
but  only  one  of  them  is  linked  to  another.  For  these  reasons,  and  because  discussion 
summarizations  have  not  yet  been  made.  Content  Synthesis  was  assigned  a  red  rating. 

Content  Organization 

Content  in  S3-XO.net  is  organized  according  to  the  fundamental  duties  of  S3s  and  XOs— 
warfighting,  training,  leadership,  administration/logistics,  and  transformation-and  includes  other 
top-level  topic  categories  addressing  the  profession — First  90  Days,  Tools  and  References,  and 
Professional  Development.  Each  top-level  topic  category  features  its  own  content,  but  also  has 
several  sub-topic  categories  constructed  around  more  specific  topics.  Some  sub-topics,  in  turn, 
have  their  own  sub-topics.  In  the  absence  of  a  formal  examination  of  the  knowledge  structures  of 
S3s  and  XOs,  there  are  no  rigorous  criteria  for  evaluating  content  organization,  but  the  current 
organization  seems  reasonable,  given  what  is  explicitly  known  about  the  professional  lives  of 
S3s  and  XOs.  Furthermore,  an  informal  review  of  the  content  in  S3-XO.net  does  not  raise  serious 
questions  as  to  why  particular  knowledge  objects  or  discussion  threads  are  loeated  where  they 
are. 


Recall  from  previous  discussion  that  of  a  small  sample  of  S3-XO.net  members  surveyed, 
respondents  (86%)  generally  indicated  that  navigating  content  topics  was  easy  or  fairly  easy. 
Respondents  (86%)  also  generally  indicated  that  they  had  no  trouble  finding  the  new  content 
additions  to  the  forum.  Respondents  (86%)  agreed  that  the  way  content  is  arranged  in  the  forum 
generally  makes  sense  or  that  they  had  not  noticed  anything  particular  about  how  the  content  was 
arranged,  good  or  bad. 

The  only  concern  raised  by  the  content  organization  was  that  it  appeared  to  be  driven 
from  the  top  down,  rather  than  from  the  bottom  up.  A  formal  assessment  of  S3/XO  knowledge 
structures  (to  include  tacit  as  well  as  explicit)  has  not  been  conducted.  Moreover,  there  were 
intermediate-level  sub-topics  that  have  very  little  content  or  no  content  at  all,  suggesting  that  the 
topic  headers  were  placed  before  the  content  accumulated  to  justify  such  headers.  Content 
distributed  across  inorganic  categories  could  be  consolidated  until  organic  categories  arise  from 
forum  activity.  The  dispersion  of  content  was  not  a  grave  concern,  save  that  it  may  have  had 
implications  for  usability  and  member  engagement.  The  software  supporting  the  online  meeting 
space  was  rather  slow,  and  the  time  required  to  move  between  multiple  topic  pages  may  have 
discouraged  browsing.  For  these  reasons.  Content  Organization  was  assigned  a  green  rating. 

Content  Activity 

Content  activity  was  difficult  to  track  because  knowledge  posts  do  not  have  upload  dates 
associated  with  them  and  download  metrics  evidently  were  not  tracked  by  Tomoye.  Approximate 
measures  of  content  activity,  as  presented  here,  must  therefore  suffice.  The  table  below  shows 
the  percentage  of  each  type  of  knowledge  post  made  by  founding  officers,  all  but  one  of  whom 
has  been  largely  inactive  in  the  forum  since  the  June  of  2003  (the  active  founding  officer’s  posts 
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are  excluded  from  the  table).  It  may  be  assumed,  then,  that  the  knowledge  posts  made  by  these 
individuals  are  approximately  two  years  old.  The  remaining  posts  are  therefore  two  years  old  or 
younger.  By  dividing  the  remaining  number  of  posts  by  96  ^  we  obtain  an  estimate  of  the 
average  weekly  rate  at  which  new  knowledge  posts  in  the  sample  have  been  added  to  the  forum. 
The  rate  projected  for  all  knowledge  posts,  because  the  sample  examined  here  is  approximately 
half  of  the  total  knowledge  posts  on  the  site,  would  be  approximately  double  the  one  shown  in 
Table  17.  This  gives  an  average  rate  of  .27  posts  a  week.  In  contrast,  one  post  every  12  hours 
would  equal  a  rate  of  14  posts  per  week,  per  topic.  There  are  eight  high-level  topics  in  S3- 
XO.net,  so  a  lower-bound  active  rate  for  content  update  is  1 12  posts  per  week.  As  shown  in 
Table  17,  the  number  of  posts  per  week  in  S3-XO.net  was  well  below  the  lower-bound  active  rate 
for  content  update.  For  this  reason.  Content  Activity  is  assigned  a  red  status  rating. 


Table  17.  Founding  Officer  Contributions  to  Knowledge  Posts 


Location 

#  of  Posts  by 
Founding  Officers 

%  of  Posts 

#  Remaining 
Posts 

#  Posts 
per  Week 

Conversations 

7 

11% 

58 

.60 

T  ool  s/Documents 

37 

60% 

25 

.26 

Website  Links 

18 

81% 

4 

.04 

Multimedia  Files 

1 

33% 

2 

.02 

Announcements 

2 

67% 

1 

.01 

Books 

0 

0% 

1 

.01 

Ideas/TTPs/Lessons  Learned 

1 

100% 

0 

0 

Content  Quality 

The  161  knowledge  posts  that  were  sampled  from  S3-XO.net  were  analyzed  for  the 
degree  to  which  their  associated  context  could  assist  the  user  in  determining  whether  the  post 
was  relevant  to  them  and  adequate  for  their  purposes*'*.  The  rubric  for  evaluating  the 
contextualization  of  website  links  was  as  follows: 

Context  was  defined  as  the  information  associated  with  the  website  link  regarding 
(1)  what  information  can  be  found  at  the  link;  (2)  the  quality  of  that  information  or 
the  conditions  under  which  the  information  can  be  of  use;  and  (3)  the  means  for 
accessing  that  information  if  the  link  does  not  work.  Website  Link  posts  were 
assigned  a  numeric  rating  of  their  contextualization,  with  0  indicating  no 
contextualization  and  3  indicating  sufficient  contextualization.  A  rating  of  3  was 
earned  if  all  three  aspects  of  context  were  addressed,  a  2  if  two  aspects  were 
addressed,  a  1  if  one  aspect  was  addressed,  and  0  if  none  of  the  aspects  was 
addressed.  Context  was  determined  not  only  from  the  information  provided  by  the 
contributor,  but  also  by  (a)  surrounding  discussion  posts,  including  comments 
associated  with  a  knowledge  post  not  made  in  the  context  of  a  conversation;  (b) 
information  gleaned  from  the  link  name;  and  (c)  inferences  that  eould  be  made 


This  number  is  derived  by  multiplying  52  times  2  and  subtracting  8,  which  is  the  approximate  number  of  weeks 
the  forum  was  shutdown  and  moved  behind  AKO. 

Note  context  ratings  should  be  independently  assigned  by  a  subject  matter  expert  and  compared  to  the  context 
ratings  presented  here.  This  step  is  necessary  to  validate  the  present  rubric. 
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about  the  quality  or  the  nature  of  the  information  at  the  link  depending  on  its 
location  (e.g.,  CALL  vs.  USA  Today). 

The  rubric  for  evaluating  the  contextualization  of  non-website  link  knowledge  posts  (i.e., 
tools/documents,  multimedia  files,  and  ideas/tactics,  techniques,  and  procedures/lessons  learned) 
is  as  follows: 

Context  was  defined  as  the  information  associated  with  the  knowledge  upload 
regarding  (1)  the  purpose/use  of  the  upload;  (2)  the  source  of  the  upload;  and  (3) 
the  quality  of  that  upload  or  the  conditions  under  which  the  upload  can  be  of  use. 

All  knowledge  uploads  were  assigned  a  numeric  rating  of  their  contextualization 
in  a  fashion  analogous  to  rating  website  link  posts.  Similarly,  context  was 
determined  by  factors  in  addition  to  the  information  provided  by  the  contributor  of 
the  upload,  including  (a)  surrounding  discussion  posts;  (b)  information  gleaned 
from  the  name  of  the  file;  and  (c)  inferences  that  could  be  made  about  the  quality 
or  nature  of  the  upload  depending  on  its  source  or  author. 

Context  ratings  for  Website  Link  posts  sampled  (N=  43)  ranged  from  0-3,  with  an 
average  of  1 .72  and  a  mode  of  2.  For  all  other  knowledge  uploads  sampled  (A^=  1 1 8),  ratings 
also  ranged  from  0-3,  with  an  average  of  1 .98  and  a  mode  of  3.  Website  Link  posts  earning 
lower  scores  often  did  so  because  they  lacked  instruction  for  accessing  the  information  provided 
at  the  link  if  the  link  did  not  work.  In  our  exploration  of  Website  Link  posts,  we  tested  19  such 
posts  and  found  that  42%  did  not  work.  For  this  reason,  such  instruction  is  critical  for  Website 
Link  posts  to  be  useful.  Other  knowledge  uploads  earning  lower  scores  often  did  so  because  they 
lacked  information  regarding  what  their  purpose/use  was.  This  occurred  more  frequently  when 
knowledge  uploads  did  not  appear  in  the  context  of  a  conversation,  but  were  posted 
independently  of  a  specific  request  or  discussion.  Knowledge  uploads  in  the  context  of 
conversation  earned  an  average  context  rating  of  2.10,  whereas  knowledge  uploads  external  to 
conversation  earned  a  context  rating  of  1 .67. 

Obviously,  this  context  assessment  is  subjective,  conducted  by  an  examiner  who  is  quite 
external  to  the  target  membership.  Contextualization  information  should  be  provided  by  the 
members  of  S3-XO.net  themselves  through  ratings  of  individual  knowledge  posts  and  feedback 
on  their  ease/difficulty  of  application.  Evidence  of  such  assessment  is  present  in  S3-XO.net 
where  members  asked  clarification  questions  about  knowledge  posts  and  where  they  indicated 
that  a  knowledge  post  was  of  particular  use.  However,  there  currently  is  no  easy,  systematic 
means  for  them  to  share  their  views  on  individual  knowledge  posts. 

One  survey  question  (see  Appendix  D  for  the  complete  survey)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  the  utility  of  content  downloads  in  the  forum.  The  majority  of 
respondents  (4)  indicated  that  they  had  found  at  least  three  downloads  in  the  forum  that  have 
been  very  helpful.  Equal  proportions  (1)  of  the  remaining  respondents  indicated  that  they  have 
found  one  useful  download  and  are  confident  there  will  be  more,  that  they  have  not  given  much 
thought  to  how  useful  the  downloads  are,  or  that  they  have  not  yet  found  a  useful  download.  For 
all  of  the  above  reasons.  Content  Quality  was  assigned  an  amber  status. 
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Professional  Links 

S3-X0.net  currently  does  not  provide  access  to  leader-development  opportunities 
that  are  either  available  outside  the  forum  [e.g.,  the  Think  Like  a  Commander  adaptive- 
thinking  training  program,  Lussier,  Shadriek,  &  Prevou  (2003),  or  simulations  or  other 
materials  used  in  instruction  at  CGSC]  or  organic  to  the  forum  (e.g.,  situational  judgment 
vignettes,  collaborative  exercises,  etc.).  For  this  reason,  Professional  Links  is  assigned  a 
red  rating. 


Conversation 


Table  18.  S3-XO.net  -  Conversation  Charaeteri sties  Status 


Conversation  Characteristics 

Status 

Red 

Amber 

Green 

Participating  Members 

Conversation  Activity 

Conversation  Quality 

Overall  Status 

Participating  Members 

As  shown  previously  in  Table  12,  34%  of  the  conversation  posts  on  S3-XO.net  have  been 
made  by  target  members.  Even  when  the  founding  officers  are  considered  target  members,  the 
percentage  of  conversation  posts  (46%)  falls  short  of  the  most  liberal  definition  of  majority  (51% 
or  more).  Twenty-four  percent  of  the  conversation  posts  was  made  by  current  S3s  or  XOs,  when 
founding  officers  are  not  considered  target  members,  and  1 7%  of  the  posts  were  made  by  eurrent 
S3s  or  XOs  when  founding  officers  are  considered  to  be  target  members.  For  this  reason. 
Participating  Members  is  assigned  a  red  status  rating.  Again,  however,  it  should  be  noted  that 
discussion  in  the  forum  might  not  represent  the  total  amount  of  discussion  that  has  occurred  due 
to  the  forum. 

Conversation  Activity 

As  of  12  May  2005,  there  were  107  unique  discussion  threads  initiated  in  S3-XO.net. 
Forty-eight  of  these  threads  (45%)  are  one-post  threads.  Of  these  48  one-post  threads,  four 
clearly  belong  to  multiple-post  threads,  but  have  somehow  become  separated,  resulting  in  1 03 
unique  discussion  threads.  The  proportion  of  post  type  in  the  remaining  44  one-post  threads  is 
compared  to  the  proportion  of  post  type  in  the  initiating  discussion  posts  for  multi-post  threads  in 
the  table  below’  .  The  rubric  for  assigning  a  type  to  a  discussion  post  is  as  follows: 

Invitation  to  Build  “New”  Knowledge  -  Invitations  to  build  “new”  knowledge  are 

general  statements  or  open-ended  questions  that  address  a  novel,  distal,  or 


This  mbric  was  developed  through  an  iterative  process  in  which  two  raters  assigned  a  post  type  to  a  sample  of  the 
1 03  initiating  posts,  collaborative  refined  the  rubric,  assigned  a  post  type  to  another  sample  of  the  initiating  posts, 
and  so  on.  Inter-rater  agreement  reached  using  the  above  rubric  was  80%.  Where  disagreement  occurred,  a  final 
post-type  assignment  was  given  on  the  basis  of  discussion  between  the  two  raters. 
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somewhat  theoretical  topic.  In  general,  these  invitations  address  organizational 
matters,  such  as  policy,  structure,  future  directions,  etc.  The  inference  is  that  the 
person  initiating  the  discussion  is  not  seeking  one  “right”  answer  as  a  basis  for 
action,  but  wishes  to  muse  collaboratively  with  the  professional  community  about 
a  broad  topic.  Asking  the  question  does  not  assume  that  other  community 
members  have  personal  experience  with  the  topic  but  that  they  are  likely  to  have 
an  educated,  professional  opinion. 

Request  for  Input  -  Requests  for  input  are  questions  that  seek  to  gather  a  range  of 
opinions  about  or  experiences  with  a  particular  topic,  generally  a  proximal  matter 
of  technical  or  tactical  procedure.  The  inference  is  that  the  person  initiating  the 
discussion  assumes  that  others  in  the  community  have  personal  experience  with 
the  topic,  that  there  may  be  more  than  one  right  way  of  addressing  the  question, 
and/or  that  sampling  multiple  perspectives  is  necessary  to  construct  an  effective 
answer  to  the  question. 

Directed  Question  -  Directed  questions  seek  a  “definitive,”  even  “yes/no”  answer 
to  a  concrete  question.  Rather  than  seeking  the  collaborative  opinion  or  experience 
of  the  professional  community,  the  directed  question  is  an  attempt  to  validate  or 
augment  information  or  receive  clarification  using  the  expertise  of  someone  in  the 
professional  community  with  relevant  experience/knowledge.  The  inference  is 
that  the  person  asking  a  directed  question  assumes  that  somebody  already  has  a 
“correct”  answer  and  seeks  to  engage  that  person. 

Request  for  Knowledge  Resources  -  Requests  for  knowledge  resources  ask 
members  to  provide  tools/documents  or  other  knowledge  products  that  can  be 
applied  to  unit  functioning,  such  as  tactical  SOPs  and  training  schedules.  These 
requests  may  be  accompanied  by  a  request  for  input  or  directed  question,  but  are 
assigned  as  requests  for  knowledge  resources  when  the  apparent  emphasis  is  on 
receiving  tools,  rather  than  opinions,  experiences,  or  instruction. 

Request  for  Expert  -  Requests  for  expert  ask  members  to  refer  someone  who  has 
particular  knowledge  in  a  specific  area.  The  inferenee  is  that  the  person  making 
the  request  assumes  that  someone  in  the  forum  is  or  knows  an  expert  in  a 
particular  area.  If  a  request  for  expert  is  accompanied  by  a  request  for  input,  a 
directed  question,  or  a  request  for  knowledge  resources,  emphasis  is  placed  on 
these  other  modes  of  inquiry  rather  than  on  the  request  for  expert. 

Information  Post  -  Information  posts  do  not  ask  a  question,  but  provide 
information  about  a  topic.  This  information  may  be  a  summary  of  some  kind, 
and/or  may  include  either  a  link  to  a  website,  a  tool/document  to  download,  or  a 
description  of  an  external  knowledge  resource.  Generally,  the  Information  Post 
contextualizes  the  knowledge  resources  to  which  it  refers.  However,  if  an 
information  post  is  accompanied  by  a  request  for  input,  a  directed  question,  or  an 
invitation  to  build  “new”  knowledge,  emphasis  is  placed  on  these  other  modes  of 
inquiry  rather  than  on  the  fact  that  information  was  posted. 
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Unknown  -  Unknown  initiating  posts  appear  to  be  an  answer  to  someone  else’s 
question  or  comment,  but  they  cannot  be  clearly  linked  to  any  of  the  extant 
conversation  threads.  These  posts  may  address  by  name  the  unknown  inquirer  to 
whom  they  are  responding  or  may  indicate  agreement  or  disagreement  with  an 
unknown  conversation  partner. 


Table  19.  Type  of  Initial 

ing  Posts  for  One-Post  and  Multi-Post  Discussion  Threads 

Post  Type 

#  Initiating 

1  -Post  Threads 

%  of  Total 

#  Initiating 
Multi-Post  Threads 

%  of  Total 

Invitation  to  Build  “New” 
Knowledge 

1 

2% 

6 

10% 

Request  for  Input 

6 

14% 

17 

29% 

Directed  Question 

7 

16% 

10 

17% 

Request  for  Knowledge 
Resources 

7 

16% 

9 

15% 

Request  for  Expert 

0 

0% 

2 

3% 

Information  Post 

20 

45% 

14 

24% 

Unknown 

3 

7% 

1 

2% 

Total 

44 

100% 

59 

100% 

Fifty-seven  percent  of  the  1 03  initial  discussion  posts  were  followed  by  a  response.  Sixty- 
eight  percent  of  the  initial  discussion  posts  that  were  explicit  requests  for  conversation  (i.e., 
invitations  to  build  “new”  knowledge,  requests  for  input,  directed  questions,  requests  for 
knowledge  resources,  and  requests  for  expert)  were  followed  by  a  response.  This  moderate  rate 
of  response  to  questions  posed  in  the  forum  combined  with  the  fact  that  only  28%  of  all  posts 
was  followed  by  another  post  within  24  hours  (and  none  of  these  was  an  initial  post),  resulted  in 
a  red-amber  status  rating  for  Conversation  Activity. 

Note,  however,  that  there  are  already  almost  as  many  discussion  posts  between  1  January 
2005  and  12  May  2005  as  there  were  in  the  entire  year  of  2003.  This  increase  in  posts  bodes  well 
for  the  Conversation  Activity  in  S3-XO.net.  However,  only  46%  of  the  initial  discussion  posts 
made  in  this  time  have  been  followed  by  a  response,  and  only  24%  of  these  posts  were  followed 
by  another  post  within  24  hours  (none  of  the  posts  followed  by  another  post  within  24  hours  was 
an  initial  post).  None  of  the  promises  to  post  additional  information,  replies,  etc.,  made  by 
discussion  participants  were  fulfilled  through  the  forum,  nor  have  any  clarification  questions 
been  answered  in  the  forum.  This  reflects  a  decrease  in  the  conversation  activity  from  the 
previous  two  years  (compare  46%  response  rate  in  2005  with  57%  in  2003).  Responding  to 
explicit  requests  for  conversation  also  decreased  from  68  to  58%. 

Conversation  Quality 

In  general,  high  standards  of  professionalism  have  been  maintained  in  S3-XO.net.  Of  the 
341  discussion  posts  reviewed,  only  one  post  was  found  to  have  an  arguable  level  of 
professionalism.  This  post  was  a  referral  to  a  debate  going  on  in  another  conversation  thread. 

This  referral  would  qualify  as  a  helpful  link  to  related  conversation,  except  that  the  conversation 
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designated  by  the  referral  was  completely  unrelated  to  the  conversation  in  which  the  referral  was 
situated.  This  gave  the  referral  the  feel  of  a  pop-up  ad. 

As  indicated  in  Table  19  above,  59  of  the  conversations  started  in  the  forum  as  of  12  May 
2005  have  resulted  in  conversation  threads  featuring  two  or  more  posts.  These  59  conversations 
were  analyzed  for  the  type  of  posts  involved  and  for  the  flow  of  conversation  succeeding  the 
initial  post.  The  one  multiple-post  conversation  thread  that  began  with  an  “Unknown”  initiating 
post  type  was  not  analyzed  because  it  was  unclear  how  the  conversation  should  flow  from  such  a 
post.  Additionally,  a  conversation  initiated  by  an  information  post  was  not  analyzed  because  the 
only  other  post  involved  was  an  advertisement  for  an  unrelated  thread  (see  the  above  paragraph). 
The  remaining  57  conversations  were  analyzed  in  detail.  In  the  absence  of  definitive  tools  for 
analyzing  discussion  content  in  knowledge-sharing  forums,  we  sampled  from  the  scholarly 
literature  where  appropriate. 

Invitations  to  Build  "New"  Knowledge.  Six  posts  initiating  multiple-post  threads  were 
invitations  to  build  “new”  knowledge.  Quality  conversation  following  an  invitation  to  build 
“new”  knowledge  involves  collectively  constructing  understanding  or  perspective  that  has  not 
previously  been  achieved.  The  product  of  the  conversation  is  collectively  “owned.”  That  is, 
discussants  should  come  away  from  the  conversation  with  the  feeling  that  they  have  participated 
in  creating  something  new— knowledge  that  was  not  possessed  by  any  of  the  discussants  prior  to 
the  interaction.  This  requires  that  discussants  share  and  discuss  perspectives  on  information, 
identify  and  explore  areas  of  disagreement,  participate  in  the  negotiation  of  new  meaning,  test 
and  modify  new  meaning,  and  come  to  consensus  on  new  meaning  (Gunawardena,  Lowe,  & 
Anderson,  1998). 

Five  of  the  six  invitations  received  more  than  one  response.  In  two  of  the  conversations, 
discussants  contributed  knowledge  products  to  aid  the  diseussion.  Importantly,  four  of  the 
conversations  involved  discussants  interacting  with  each  other,  commenting  on  or  building  from 
each  other’s  contributions.  In  addition,  two  of  the  six  conversations  featured  a  comment  on  the 
direction  of  the  conversation  by  one  of  the  discussants.  However,  none  of  the  conversations 
featured  a  post  that  closed  the  loop,  indicating  that  the  discussants  believed  the  knowledge- 
creation  process  had  been  completed  or  that  something  had  been  learned  from  the  process.  Four 
of  the  six  conversations  involved  individual  discussants  making  multiple  contributions,  but  only 
in  one  case  did  more  than  one  person  make  multiple  posts  to  the  conversation.  It  appears  that 
invitations  to  build  “new”  knowledge  can  be  active  (average  thread  length  was  6  posts),  but  also 
that  they  seem  to  stop  before  new  knowledge  is  constructed. 

Requests  for  Input.  Seventeen  of  the  initiating  posts  were  requests  for  input.  Quality 
conversation  following  a  request  for  input  involves  the  sharing  of  multiple  perspectives  and 
achieving  a  more  complete,  or  representative,  understanding  of  a  complex  phenomenon.  This 
requires  that  the  discussion  have  multiple  participants  constructively  sharing  their  perspectives 
and  a  synthesis  of  these  perspectives  that  illuminates  the  trends  in  the  information  or  attitudes 
shared. 


Twelve  requests  for  input  received  multiple  responses  (between  two  and  nine),  and  nine 
spawned  17  related  questions.  Four  of  the  related  questions  were  answered,  and  two  received 
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comments.  None  of  the  three  clarification  questions  raised  were  answered.  Five  of  the 
conversations  featured  comments  on  aspeets  of  the  discussion,  two  of  these  conversations 
featuring  multiple  comments.  As  with  invitations  to  build  “new”  knowledge,  requests  for  input 
engaged  active  responding  (7  posts  on  average).  However,  as  was  the  case  previously,  the 
synthesis  and  dialogue  critical  to  building  new  understanding  generally  did  not  take  place. 

Directed  Questions.  Ten  initiating  posts  were  directed  questions.  Quality  conversation 
following  a  directed  question  involves  answering  the  question  posed.  The  asker  should  come 
away  from  the  discussion  with  actionable  knowledge  that  he  did  not  previously  have.  This 
requires  articulating  one’s  question  to  elicit  the  required  answer  and  engaging  discussants  to 
provide  information  until  a  satisfactory  answer  is  reached. 

All  ten  questions  were  answered  either  by  a  direct  answer  (6)  or  by  reference  to 
knowledge  resources  (4).  Two  questions  received  multiple  answers  and  two  featured  comments 
on  aspects  of  the  discussion.  One  of  these  comments  was  on  the  discussion  itself  and  one 
involved  a  summarization  of  what  was  said  in  the  preceding  posts.  Two  clarification  questions 
were  asked,  but  none  was  answered.  Three  conversations  spawned  four  related  questions,  two  of 
which  were  answered.  Two  of  the  directed  questions  received  offers  of  assistance.  Here  too, 
members  appeared  responsive  to  other  members’  needs,  although  the  dialogue  necessary  to 
ensure  the  answer  is  maximally  relevant  (e.g.,  answering  clarification  questions)  did  not  occur. 
None  of  the  ten  conversations  featured  a  post  that  closed  the  loop,  indicating  that  the  question 
initially  asked  had  been  answered. 

Requests  for  Knowledge  Resources.  Nine  of  the  initiating  posts  were  requests  for 
knowledge  resources.  Quality  conversation  following  a  request  for  knowledge  resources  involves 
referring  the  requestor  to  the  knowledge  resources  he  seeks  and  ensuring  the  requestor  can  access 
and  apply  the  resources  referred  to  him.  This  requires  delivery  of  or  direction  to  resources  and 
providing  context  for  applying  these  resources. 

All  of  the  nine  requests  were  responded  to  either  by  one  or  more  references  to  knowledge 
resources  (5  requests),  offers  of  assistance  (2),  or  some  combination  of  the  two  (2).  Six  requests 
received  between  two  and  five  references  to  knowledge  resources.  Three  conversations  featured 
comments  on  the  requests  or  the  knowledge  resources  referred.  Requests  for  knowledge 
resources  appeared  to  have  engaged  productive  responding.  However,  none  of  the  nine 
conversations  featured  a  post  that  closed  the  loop,  indicating  that  the  initial  need  had  been  met. 

Requests  for  Experts.  Two  of  the  posts  initiating  multiple-post  threads  were  requests  for 
experts.  Quality  conversation  following  a  request  for  an  expert  involves  providing  the  requestor 
with  a  referral.  In  both  conversations,  a  referral  was  made,  and  one  request  received  an  offer  of 
assistance.  As  with  requests  for  knowledge  resources,  requests  for  experts  appeared  to  result  in 
productive  interaction  with  S3-XO.net  members  and  facilitators.  Here  too,  however,  none  of  the 
requests  for  experts  featured  a  close-the-loop  post  indicating  that  the  requester  made  the 
connection  he  needed. 

Information  Posts.  Thirteen  of  the  posts  initiating  multiple-post  discussion  threads  were 
information  posts.  Quality  conversation  following  an  information  post  involves  questioning  and 
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commenting  that  contextualizes  the  information  provided,  gauging  its  quality  and  applicability  to 
potential  users. 

Eight  of  the  information  posts  received  comments  from  S3-XO.net  members  on  the  post 
itself  or  on  the  topic  of  the  post  more  generally.  Two  information  posts  received  multiple 
comments.  One  highly  decontextualized  information  post  received  a  clarification  question,  but  it 
was  not  answered.  Eight  of  the  information  posts  spawned  nine  related  questions,  three  of  which 
were  answered.  Overall,  information  posts  did  not  generate  a  great  deal  of  discussion.  The 
average  length  of  discussion  threads  initiated  by  information  posts  was  3  posts  (including  the 
initiating  post).  It  is  possible  that  conversation  was  not  necessary  to  sufficiently  contextualize  the 
information  posts,  but  attempts  by  S3-XO.net  members  to  use  the  conversation  for  learning  (i.e., 
through  related  questions)  generally  did  not  engage  other  members. 

One  survey  question  (see  Appendix  D  for  the  complete  survey)  asked  S3-XO.net 
members  (see  footnote  #8  in  Joining  the  Forum  for  a  description  of  the  survey  sample)  to 
indicate  their  impressions  of  the  utility  of  conversations  in  the  forum.  The  modal  number  of 
respondents  (4)  indicated  that  they  had  read  one  valuable  conversation  in  the  forum  and  that  they 
are  confident  there  would  be  more.  Roughly  equal  proportions  of  the  remaining  respondents 
indicated  that  they  have  read  at  least  three  valuable  conversations  (2)  or  that  they  had  not  yet 
read  a  valuable  discussion  (1).  For  these  reasons,  Conversation  Quality  was  assigned  a  red-amber 
status. 


Social  and  Intellectual  Capital  -  Recommendations 
Connections 

Membership  composition  is  key  to  generating  productive  activity  in  the  forum  (Stuckey 
&  Smith,  2004;  Wenger  et  al.,  2002),  and  S3-XO.net  requires  a  significant  increase  in  target 
members.  Rather  than  focusing  recruitment  efforts  on  CGSC  students,  greater  emphasis  should 
be  placed  on  recruiting  current  S3s  and  XOs  through,  for  example,  advertisements  of  the  site  on 
installation  homepages  and  on  other  discussion  forums  (including  the  informal  chat  forums  in 
AKO),  exchange  with  National  Training  Center  observer/controllers,  and  visits  to  installations. 
The  upcoming  membership  drive  planned  by  the  S3-XO.net  support  personnel  is  a  step  in  the 
right  direction,  but  more  active  recruiting  of  current  S3s/XOs  may  be  necessary.  Once  members 
have  been  recruited,  it  may  be  helpful  to  make  it  a  requirement  to  supply  biographical 
information,  especially  information  about  assignment  history.  Ideally,  new  members  will  see 
themselves  as  contributors  to  the  forum  and  voluntarily  provide  this  information,  but  currently 
approximately  48%  of  new  target  members  do  so.  Yet,  according  to  our  interviews  with  current 
and  potential  members  of  S3-XO.net,  this  information  is  critical  in  establishing  the  credibility  of 
post  authors.  Our  interviews  also  indicated  that  the  target  audience  of  the  forum  is  highly 
desirous  of  being  able  to  locate  the  S3s/XOs  in  the  units  they  are  assigned  to  for  their  next 
S3/XO  duty  position.  Locating  these  officers,  keeping  their  contact  information  current  in  the 
forum,  and  ideally,  recruiting  these  officers  to  participate  in  the  forum  would  represent  a  difficult 
but  very  important  step  toward  facilitating  the  development  of  connections. 
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Context 

The  opportunities  to  develop  shared  narratives  in  S3-XO.net  seem  relatively  infrequent 
and  sparsely  attended.  Alternative  methods  to  book  reviews  for  creating  shared  experiences  may 
prove  to  be  more  engaging  if  they  take  less  time  and  are  more  directly  related  to  the  jobs  of  the 
S3/XO.  These  methods  might  include  product  critiques  (e.g.,  tactical  operations  center  setups, 
training  plans,  etc.)  or  professional  article  critiques  (e.g.,  publications  in  Armor  magazine. 
Military  Review,  or  Parameters).  If  they  are  carefully  created,  command  challenges  such  as  those 
found  on  CC.mil  may  also  generate  discussion  in  a  direction  more  closely  related  to  matters  of 
leadership,  interpersonal  relationships,  and  ethics.  Facilitators  should  use  their  own  social  capital 
to  arrange  for  multiple  members  to  participate  in  a  shared  experience  at  one  time  in  order  to 
generate  activity  and  interest.  Simply  posting  the  opportunity  may  not  be  enough,  as  the 
relatively  sparse  discussion  surrounding  other  knowledge  or  information  posts  indicates. 

Content 

Content  activity  is  sluggish,  and  mostly  driven  by  non-target  members  of  the  forum. 
Moreover,  four  percent  (approximately  7)  of  the  1 57  contributions  sampled  were  made  by  a 
current  S3  or  XO.  There  are  multiple  possible  explanations  for  why  there  are  relatively  few 
knowledge  postings  to  the  site,  especially  by  current  S3s/XOs,  including  (a)  S3s/XOs  with 
limited  time  only  post  knowledge  when  it  has  been  directly  requested;  (b)  S3s/XOs  feel  reluctant 
to  share  their  knowledge  either  out  of  a  sense  of  competition,  anti-intellectualism,  or  concerns 
about  criticism;  or  (c)  current  S3s/XOs  do  not  see  themselves  as  contributors  to  the  forum. 
Contact  and  encouragement  by  facilitators  may  help  increase  posting  by  target  members,  if  these 
members  are  not  posting  for  the  above  reasons  (Dixon  et  al.,  2005). 

In  contrast,  content  organization  and  contextualization  in  S3-XO.net  appear  to  be 
effective.  However,  some  minor  changes  could  increase  the  ease  by  which  content  is  accessed 
and  reviewed  for  relevance.  First,  the  front  page  of  the  forum  should  be  changed  frequently  by 
featuring  new  content,  new  members,  or  new  points  of  interest  for  the  S3/XO.  Recent  changes  to 
the  front  page  of  the  forum  suggest  a  hopeful  step  in  the  right  direction  provided  that  follow-on 
changes  do  not  take  four  months  to  occur.  Second,  related  content  should  be  consolidated  such 
that  discussions  or  knowledge  objects  on  the  same  subject  are  not  distributed  throughout  the 
forum  unless  they  are  distributed  as  a  whole.  Third,  duplicate  posts  and  duplicate  headers  should 
be  removed  (e.g.,  there  are  two  Rear  Detachment  Operations  folders  inside  Admin/Log; 
Leadership  has  an  Ethics  and  Morals  Folder  and  a  Morals  and  Ethics  folder).  Topic  folders 
without  posts  should  be  removed.  Finally,  unless  metrics  and  counters  provided  by  Tomoye  can 
be  explained  or  repaired,  they  should  be  removed.  A  couple  of  changes  could  enhance  the 
eontextualization  of  knowledge  objects,  including  reducing  the  number  of  clicks  required  to  get 
to  comments  on  knowledge  objects,  featuring  “date/time  posted”  information  for  all  knowledge 
objects,  and  adding  an  enhanced  rating  capability  such  that  individual  objects  can  be  rated  and 
their  ratings.  Ratings  and  date/time  posted  information  should  be  viewable  before  the  object  is 
clicked  on. 

Conversation 

The  implication  of  several  interviews  we  conducted  is  that  the  role  of  facilitators  is 
especially  critical  in  a  discussion  forum  that  is  geared  towards  S3s/XOs.  The  current  S3/XO  who 
would  login  to  S3-XO.net  is  several  years  older  than,  for  example,  the  current  company 
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commander  logging  into  CC.mil.  S3s/XOs  are  more  experienced  with  the  Army  as  an 
organization,  and  therefore  have  more  confidence  in  their  place  in  the  organization  and  in  their 
ability  to  solve  problems.  S3s/XOs  who  are  in  the  National  Guard  have  the  additional  experience 
of  having  held  down  a  successful  career  outside  of  the  Army.  For  this  reason,  they  may  feel  less 
need  for  affiliation/validation  than  company  commanders.  Among  S3s/XOs  in  the  active  Army, 
there  is  a  strong  sense  of  competition  for  coveted  command  positions.  S3s/XOs  also  deal  with 
problems  much  larger  in  scope  than  company  commanders.  They  must  delegate  projects,  rather 
than  tasks,  and  must  deal  with  combined  arms  tactics  and  training.  This  makes  them  busier  than 
ever  (working  12-hour  days  and  weekends)  at  a  time  when  they  are  more  likely  to  have  families 
who  place  demands  on  their  time  outside  of  work.  The  greatest  incentive  for  an  S3/XO  to 
participate  in  the  forum,  then,  is  confidence  that  the  forum  will  get  him  what  he  needs  when  he 
needs  it  and  obligation  to  give  back  what  he  has  received.  Facilitators  must  provide  this  incentive 
by  ensuring  that  responses  to  member  questions  are  timely  and  adequate. 

Conversation  in  S3-XO.net  is  sluggish  and  it  is  unclear  how  useful  the  productive 
conversation  is  to  members.  Shorter  times  between  posts  not  only  would  reflect  greater  activity, 
but  also  will  generate  greater  activity.  To  increase  the  perception  of  activity  in  the  conversation 
threads,  discussions  beginning  and  ending  in  2003  or  2004  should  be  archived  and  made  easy  to 
reference.  In  addition,  requests  or  questions  for  which  there  are  multiple  posts  should  feature 
some  kind  of  follow  up  that  closes  the  loop  indicating  how  the  request  or  question  was  addressed 
(see  Dixon  et  al.,  2005).  Fewer  conversations  starting  with  references  to  knowledge  resources  or 
information  posts  may  generate  more  discussion  unless  the  forum  staff  can  arrange  to  have 
multiple  members  generate  and  sustain  conversation  activity  around  such  posts.  Facilitators  must 
actively  link  members  with  questions  to  members  or  other  people  with  answers  through 
exploration  of  the  membership  or  making  use  of  their  own  social  capital  (Dixon  et  al.,  2005). 
Facilitators  must  use  these  connections  to  ensure  that  every  question  has  some  level  of  response 
(not  just  acknowledgement)  within  24  hours. 


Overall 

The  software  supporting  the  S3-XO.net  online  meeting  space,  Tomoye  Simplify,  is 
generally  recognized  as  the  platform  of  choice  for  online  communities  of  practice,  at  least  in  the 
military  (Glennie  &  Hickok,  2003;  Kendall  &  McHale,  2003).  Tomoye  is  preferred  by  the 
military  because  it  is  more  cost-effective  than  other  software  solutions  available  (including  a 
solution  developed  in-house)  and  relatively  easy  to  use  both  by  facilitators  and  forum  members. 
Importantly,  it  provides  several  of  the  functionalities  key  to  effective  collaboration,  including 
asynchronous  chat,  search  capability,  and  member  databases. 

Unfortunately,  one  limitation  of  Tomoye  is  that  it  is  slow  and  it  is  often  difficult  to  wait 
for  new  pages  to  appear.  Effort  should  be  made  to  reduce  the  number  of  steps  members  must  go 
through  to  find  particular  items  or  simply  to  browse.  Otherwise,  members  may  lose  patience  and 
disengage  from  the  forum.  Some  relatively  straightforward  changes  that  could  address  this 
limitation  are  as  follows: 

1)  Provide  information  in  the  post  initiating  a  conversation  that  indicates  whether  the 
conversation  contains  knowledge  objects  and  when  the  most  recent  post  to  the  conversation  was 
made. 
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2)  Show  all  of  the  sub-topics  associated  with  a  particular  top-level  topic  even  when  the 
page  opened  is  in  a  sub-topic.  This  way,  members  do  not  have  to  return  to  the  top-level  topic  to 
view  another  sub-topic. 

3)  Add  a  “browse  knowledge”  capability  so  that  members  can  see  a  list  of  all  of  the 
knowledge  objects  at  one  time,  organized  by  topic/subtopic  and  with  links  to  related  discussion 
posts,  including  comments. 

S3-XO.net  Impact  on  Member  Competence  and  Professionalism 

Recall  from  earlier  in  this  report  that  in  order  to  manage  the  scope  of  individual-level 
impact  assessment  one  should  target  those  personal  characteristics,  competencies,  and  actions  for 
which  common  or  persistent  problems  have  been  identified  in  the  target  membership.  In  the  case 
of  S3-XO.net,  the  target  membership  is  comprised  of  current  and  future  S3s  and  XOs.  Individual- 
level  development  should  therefore  prepare  future  S3s/XOs  to  assume  their  next  duty  position 
and  should  support  current  S3s/XOs  as  they  perform  their  job  duties. 

Recall  also  that  the  feasibility  of  the  quasi-experimental  approach  to  assess  individual- 
level  impact  is  highly  questionable.  A  reasonable  proxy  is  to  assess  whether  or  not  the  content 
and  discussion  in  S3-XO.net  addresses  the  key  problem  areas  in  S3/XO  competence  and 
professionalism.  The  social  and  intellectual  capital  assessment  criteria  applied  above  could  be 
used  to  assess  the  functional  effectiveness  of  the  forum  content  in  the  key  areas  of  interest. 

For  these  reasons,  the  following  section  presents  an  overview  of  the  personal 
characteristics,  competencies,  and  actions  expected  of  S3s/XOs,  highlighting  general  areas  where 
difficulty  has  been  identified.  We  used  a  combination  of  literature  review  and  interviews  with 
subject  matter  experts  to  understand  organizational  expectations  of  the  S3/XO  and  to  identify 
areas  of  difficulty.  A  list  of  the  types  of  subject  matter  experts  we  interviewed  is  provided  in 
Appendix  A.  If  S3-XO.net  is  to  have  an  impact  on  officer  competence  and  professionalism,  and, 
by  extension,  unit  and  organizational  effectiveness,  these  areas  should  represent  a  significant 
proportion  of  the  productive  discussion  and  knowledge  sharing  occurring  in  the  forum. 

Requirements  of  the  S3/XO 

The  S3/XO  duty  position  is  a  critical  point  in  an  Army  officer’s  career  because  the 
position  that  he  assumes  after  being  an  S3/XO  will  set  the  tone  for  the  rest  of  his  career.  That  is, . 
his  perfonuance  as  an  S3/XO  is  a  critical  determinant  of  the  Army’s  decision  to  select  him  for 
promotion  and  command.  At  this  point  in  his  career,  the  Army  officer  generally  has  decided  to 
remain  in  the  Army  until  retirement.  In  addition,  the  Army  has  invested  in  him,  having  chosen 
him  to  go  to  CGSC  as  preparation  to  become  an  S3/XO  and  then  choosing  him  for  a  position  that 
is  a  critical  stepping-stone  to  command.  Although  the  S3/XO  does  not  have  to  juggle  the 
demands  of  command  and  administration,  as  does  the  company  commander,  he  must  work 
extremely  hard  to  compete  with  the  other  highly  qualified  S3s/XOs  for  limited  command 
positions  and  continued  upward  mobility  in  the  Army.  The  future  S3s/XOs  we  interviewed 
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expressed  that  an  S3/X0  community  that  could  help  them  save  time  and  increase  productivity, 
develop  camaraderie,  and  seek  advice  or  advise  others  would  be  highly  desirable. 

As  field  grade  officers  and  leaders  in  the  U.S.  Army,  S3s  and  XOs  are  expected  to 
embody  certain  values  and  personal  characteristics,  demonstrate  particular  competencies,  and 
execute  specific  actions.  These  values,  competencies,  and  actions  correspond  to  the  Army’s  Be, 
Know,  Do  conceptualization  of  leadership  [Field  Manual  (FM)  22-100;  U.S.  Department  of  the 
Army  (DOA),  1999],  and  serve  as  the  foundation  for  effective  units  and,  ultimately,  an  effective 
organization.  Participation  in  S3-XO.net  has  the  potential  to  foster  officer  attitudes,  capabilities, 
and  skills  that  meet  or  exceed  the  expectations  of  unit  commanders  and  the  organizational  Army 
not  only  through  knowledge  sharing  but  also  through  relationship  building  that  fosters  a  sense  of 
mutual  trust,  identification,  and  obligation  within  the  professional  community. 

Values  -Being 

The  Army  values  described  in  FM  22-100  are  loyalty,  duty,  respect,  selfless  service, 
honor,  integrity,  and  personal  courage.  Soldiers  are  expected  to  be  loyal  to  the  U.S.  Constitution, 
to  the  nation’s  civilian  leadership,  to  the  Army,  to  the  unit  they  serve,  and  to  other  Soldiers.  They 
are  to  put  the  welfare  of  the  nation,  of  the  Army,  and  of  other  Soldiers  ahead  of  their  own.  They 
are  expected  to  fulfill  their  obligations  to  the  best  of  their  ability  and  to  treat  people  in  the 
marmer  in  which  they  wish  to  be  treated.  They  must  live  up  to  the  Army  values,  making  legal, 
moral,  and  ethical  choices  even  in  the  face  of  fear,  danger,  or  adversity.  Adherence  to  these 
values  constitutes  an  important  aspect  of  Army  professionalism-the  conduct,  aims,  or  qualities 
that  characterize  the  professional  leader. 

Several  environmental  conditions  currently  threaten  the  level  of  professionalism  achieved 
in  the  Army  after  the  Vietnam  War.  One  condition  is  the  tension  between  post-modern  relativism 
predominant  in  civilian  culture  and  the  objective  professional  ethic  expected  by  the  Army 
(Snider,  Nagl,  &  Pfaff,  1999).  A  second  condition  is  the  challenge  to  the  Army’s  and  Soldiers’ 
warfighting  identity  introduced  by  full-spectrum  operations  and  the  increased  prevalence  of 
unconventional  attacks  against  national  interests  (e.g.,  computer  hacking)  that  places  civilian 
experts  in  roles  of  active  national  defense  (Gray,  2001).  Challenges  to  the  unique  purpose  and 
legitimacy  of  the  Army  as  the  nation’s  warfighters,  which  serve  as  the  basis  of  professionalism 
(Snider  et  al.,  1999),  threaten  adherence  to  Army  values,  especially  in  a  time  when  the  objective 
professional  ethic  has  come  under  question.  Moreover,  doctrine  describes  Army  values,  but 
knowing  what  these  values  are  is  not  sufficient  to  make  decisions  that  adhere  to  these  values, 
particularly  when  a  decision  requires  giving  one  value  precedence  over  another  (Patterson  & 
Phipps,  2002).  Increased  ethnic  and  cultural  diversity  in  the  Army  presents  further  challenges  to 
maintaining  an  objective  professional  ethic  and  arriving  at  a  unified  approach  to  moral  decision¬ 
making  on  the  basis  of  an  overview  description  of  values. 

Some  recent  reports  suggest  that  adherence  to  Army  values  is  problematic  and  has 
negative  implications  for  retention  and  the  quality  of  the  retained  force.  For  example,  Bilafer 
(2001)  claims  that  senior  officers  in  the  Army  are  failing  to  adhere  to  Army  values  and  suggests 
that  this  failure  to  maintain  rigorous  standards  for  behavior  is  having  negative  effects  on  the 
morale  of  junior  and  intermediate  officers,  with  troubling  implications  for  retention,  cynicism. 


73 


and  careerism  (see  also  Marlin,  1997;  Wong,  2000).  For  eaeh  Army  value,  Bilafer  (2001) 
presented  stories  illustrating  the  importance  that  some  senior  commanders  have  placed  on  self- 
promotion  over  good  eonduct.  These  stories  feature  absenee  of  top-down  loyalty,  lack  of  care  for 
Soldiers,  including  inadequate  mentorship  [see  also  Army  Training  and  Leader  Development 
Panel  (ATLDP),  2003],  inconsiderate  behavior  toward  subordinates,  ineluding  harassment  and 
intimidation,  and  misrepresentation  of  unit  readiness.  Although  this  report  is  now  more  than  four 
years  old,  it  serves  as  an  alert  to  the  effeets  that  threats  to  professionalism  can  have  on  leader 
behavior. 

Another  recent  study  (Patterson  &  Phipps,  2002)  reports  that  the  number  of 
eommissioned  officers  separated  from  the  Army  for  ethieal  miseonduet  (as  opposed  to  other 
causes)  increased  by  11%  between  1992  and  2000,  from  1.5%  to  3.9%  of  all  separations.  This 
study  likely  underestimates  the  aetual  occurrence  of  ethical  misconduct  by  Army  officers 
because  many  instances  of  ethical  misconduct  may  never  come  to  light.  In  addition,  the 
aftermath  of  recent  (Abu  Ghraib,  Bagram)  and  historical  (My  Lai)  ethical  scandals  suggests  that 
ethical  misconduct  conducted  by  officers  or  approved  by  officers  that  does  come  to  light  may  not 
necessarily  result  in  separation  of  officers  from  the  Army  (though  see  “2  Officers  Punished  in 
2003,”  2005). 

In  eontrast  to  these  findings,  Wong  (2000)  and  the  Operation  Iraqi  Freedom  Mental 
Health  Advisory  Team  (OIF  MHAT,  2003)  have  both  reported  high  levels  of  professional 
behavior  or  feelings  of  professionalism  among  officers  even  in  the  midst  of  grave  job 
dissatisfaction  and  low  morale.  It  is  difficult  to  know  without  comprehensive  study  how 
pervasive  unethical  or  unprofessional  behavior  in  the  Army  is,  let  alone  the  impact  that  this 
behavior  has  on  unit  and  organizational  effectiveness.  Assessing  the  “true”  status  of  Army 
professionalism  is  difficult  in  part  because  Army  professionalism  is  not  necessarily  well 
understood  (Gray,  2001)  or  measured,  particularly  now  as  the  Army’s  fundamental  role  is  in  the 
midst  of  change. 

In  any  ease,  threats  to  organizational  identity  and  individual  responsibility— especially 
full-spectrum  operations— are  real  and  affect  not  only  commissioned  officers  but  the  Soldiers 
they  lead  (Britt,  1998).  The  morale  of  Army  Soldiers,  particularly  those  serving  in  Operation 
Iraqi  Freedom  is  quite  low,  with  suicide  rates  among  Soldiers  higher  than  in  any  other  recent 
conflict,  including  Vietnam  (OIF  MHAT,  2003).  Moreover,  it  may  be  argued  that  in  an 
organization  with  an  objective  professional  ethic  just  one  person  who  strays  from  that  ethic  just 
one  time  represents  a  significant  threat  to  the  moral  fiber  of  the  organization  as  a  whole.  Field 
grade  officers  therefore  must  be  eonstantly  mindful  of  their  adherenee  to  Army  values  and  must 
also  ensure  that  the  Soldiers  they  lead  are  supported  in  their  efforts  to  do  the  same. 

Partieipation  in  an  APF  may  help  field  grade  officers  defend  against  threats  to 
professionalism  through  exposure  to  and  bonding  with  eolleagues  in  a  professional  setting.  Such 
activity  could  expose  officers  to  positive  or  even  inspiring  examples  of  professionalism,  allowing 
social  pressures  to  shape  moral  and  ethical  behavior.  Similarly,  such  exposure  may  buttress  the 
professionalism  of  those  officers  who  experience  a  lack  of  professionalism  in  their  unit  and  are 
coneemed  that  it  is  typical  of  the  Army  as  a  whole.  Sharing  knowledge  in  the  professional  forum 
may  also  provide  an  opportunity  for  officers  to  collaboratively  define  the  unique  purpose  and 
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legitimacy  of  their  own  roles  in  supporting  their  unit,  especially  where  transformation  and  full- 
spectrum  operations  have  required  role  re-definition.  In  order  for  participation  in  an  APF  to  have 
this  kind  of  benefit,  discussion  must  address  questions  of  professional  identity,  of  values,  and  of 
ethical  and  moral  leadership.  Discussion  must  include  stories  and  personal  reflections,  which  are 
vehicles  for  sharing  tacit  knowledge  necessary  for  successful  leadership  (Cianciolo,  Antonakis, 

&  Sternberg,  2004).  To  support  the  S3  and  XO  in  specific,  discussion  must  also  address  the 
personal  characteristics  staff  officers  are  expected  to  embody. 

FM  6-0  (DOA,  2003)  identifies  nine  personal  characteristics  continually  developed  by 
effective  staff  officers:  competent,  proactive,  creative,  flexible,  self-confident,  loyal,  team- 
oriented,  effective  manager,  and  effective  communicator.  Some  of  these  personal  characteristics 
overlap  in  obvious  ways  with  Army  values,  but  others  are  specified  more  in  terms  of  the 
requirements  of  staff  officers  in  particular.  These  are  areas  of  special  challenge  for  most  staff 
officers,  including  the  S3  and  XO. 

For  example,  the  competent  staff  officer  as  described  in  FM  6-0  is  familiar  with  his  own 
and  other  staff  officers’  functional  duties.  He  is  also  a  team  player  who  advises,  consults  and 
cooperates  with  other  staff  officers.  However,  staff  officers  at  the  battalion  and  brigade  level 
have  consistently  shown  difficulties  with  horizontally  and  vertically  sharing  information  and 
integrating  their  efforts  (CALL,  1998,  2003).  The  successful  staff  officer  manages  time  and 
resources  effectively,  avoiding  duplication  of  effort,  but  reinventing  the  wheel  is  generally 
recognized  to  be  a  common  problem  in  the  Army.  Finally,  the  effective  staff  officer  must  be 
concise  in  his  communications,  writing  concise  orders  and  plans,  staff  studies,  staff  summaries, 
and  reports.  Staff  officers  have  particular  difficulty  in  this  area,  often  failing  to  analyze 
information  before  briefing  the  commander.  Challenges  to  efficient  and  effective  communication 
will  only  increase  as  digital  systems  provide  increased  volumes  of  information  to  sift  through 
(Langley,  2004;  White,  2001). 

Difficulties  in  these  areas  appear  to  stem  from  the  perceived  lack  of  time  available  for 
staff  officers  to  perform  immediate  tasks  while  simultaneously  establishing  a  foundation  for  the 
effective  performance  of  fiiture  tasks.  Moreover,  the  organizing  principles  that  staff  officers 
could  use  to  increase  efficiency  (e.g.,  commander’s  intent,  understanding  of  each  other’s  roles; 
Cianciolo  &  Sanders,  to  appear;  Olmstead,  1992)  are  often  lacking.  For  the  S3  and  XO  to  be 
supported  by  community  building  and  knowledge  sharing,  discussion  must  provide  a  means  to 
save  time  but  must  also  enable  the  S3/XO  to  (a)  better  conceptualize  his  role  in  the  staff,  in  the 
command  and  control  (C2)  team,  and  in  the  unit;  (b)  increase  his  awareness  of  the  interpersonal 
as  well  as  technological  means  available  to  him  to  manage  information  effectively;  (c) 
internalize  his  role  as  a  leader  and  role  model  for  his  peers  and  subordinates,  and  (d)  envision  all 
situations  as  potential  leverage  for  collective  and  personal  success. 

Competencies  -  Knowing 

FM  22-100  (DOA,  1999)  outlines  four  areas  of  leader  competence— tactical,  technical, 
conceptual,  and  interpersonal— which  broadly  characterize  what  a  leader  must  know  to  lead 
successfully.  Tactical  competencies  are  the  skills  necessary  to  conduct  operations  and  are 
enhanced  by  effective  technical,  conceptual,  and  interpersonal  competence.  Technical 
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competencies  relate  to  the  basie  tasks  and  functions  the  officer  must  perform  on  the  job. 
Coneeptual  competencies  enable  sound  analysis,  judgment,  and  decision-making.  Interpersonal 
competencies  are  those  competeneies  that  enable  the  leader  to  exert  his  influence  and  ensure  his 
will  is  carried  out. 

Tactical  Competence.  Tactical  competence  comprises  warfighting  capability  and  the 
synchronization/orchestration  of  friendly  military  and  non-military  assets  to  achieve  decisive 
results  on  the  battlefield.  As  staff  officers,  neither  the  S3  nor  the  XO  has  the  command  authority 
to  employ  units  in  combat  under  normal  circumstances.  Therefore,  to  demonstrate  tactical 
competence  staff  offieers  must  be  proficient  at  synchronizing  mission  plans  and  operations. 
Synchronization  is  defined  as  “arranging  activities  in  time,  space,  and  purpose  to  mass  maximum 
relative  eombat  power  at  a  decisive  place  and  time”  (FM  3-0,  DOA,  2001).  Aiding  the 
commander  in  planning  and  executing  missions  by  achieving  synchronization  is  the  fundamental 
tactical  role  of  staff  officers.  It  is  the  responsibility  of  the  XO  and  S3  as  leaders  of  the  staff  to 
ensure  that  the  staff  is  trained  and  integrated  such  that  they  can  synchronize  their  areas  of  interest 
effectively. 

Synchronization  involves  knowing  the  capabilities  and  limitations  of  each  battlefield 
operating  system  (BOS)  at  a  particular  time  given  a  particular  location,  integrating  the  efforts  of 
each  BOS,  and  leveraging  BOS  capabilities  to  create  a  desired  effect  on  the  enemy  or  the 
environment.  Fundamentally,  synchronization  is  difficult  because  it  requires  staff  officers  to 
anticipate  one  another’s  needs  and  communicate  freely  in  a  time-constrained  environment  with 
substantial  personnel  turnover,  diverse  experience  levels,  and  considerable  complexity  (e.g., 
digital  C2  systems,  unpredictable  enemy  activity,  transforming  tactical  roles).  Moreover,  the 
rapid  everyday  operational  tempo  experienced  by  staff  officers  can  preclude  the  development  of 
technical,  conceptual,  and  interpersonal  competence  necessary  for  conducting  collective  tactical 
tasks  as  staff  officers  struggle  just  to  keep  up  with  day-to-day  managerial  demands.  As  the  chief 
of  staff,  the  XO  must  step  back  from  immediate  day-to-day  activities  and  assess  the  staffs 
collective  capability,  identify  areas  of  strength  and  weakness,  and  train  the  staff  to  strengthen 
weak  areas  and  maintain  areas  of  strength.  In  addition,  the  XO  must  put  standing  operating 
proeedures  (SOPs)  into  place  for  how  the  staff  as  a  colleetive  will  conduct  mission  planning  and 
support  the  commander  during  execution,  which  will  aid  in  team  eommunieation  and 
information  management.  As  the  operations  officer,  the  S3  must  ensure  that  his  day-to-day 
activity,  such  as  conducting  training  and  evaluating  unit  readiness,  involves  a  systems  approach 
in  which  unit  readiness  is  conceptualized  as  the  operational  effectiveness  of  the  eombined  assets 
available  to  the  unit.  The  S3  also  assists  the  commander  in  the  execution  of  current  and  planning 
of  future  operations. 

To  support  the  development  of  tactical  competence  in  the  XO,  then,  knowledge  sharing 
and  discussion  must  provide  a  means  for  XOs  to  rapidly  prepare  staff  readiness  assessment  and 
training  plans  and  to  quickly  develop  SOPs  for  mission  planning  and  staff  coordination  during 
execution.  Knowledge  sharing  and  discussion  must  provide  a  means  for  XOs  and  S3ss  to  more 
easily  conceptualize  operations  as  combined  arms  missions  and  to  translate  the  coneeptualization 
of  eombined  arms  missions  into  effective  training  plans.  Both  S3s  and  XOs  may  demonstrate 
increased  tactical  competence  if  knowledge  sharing  and  discussion  enabled  them  to  understand 
and  address  the  implications  of  diverse  levels  of  experience  in  the  staff,  to  rapidly  form  trusting 
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relationships  with  new  staff  offieers,  and  to  manage  more  effeetively  the  demands  of  the 
eomplex  operational  environment. 

Technical  Competence.  Teehnieal  competence  comprises  awareness  and  use  of  doctrine, 
technological  exploitation,  understanding  of  how  diverse  conditions  affect  unit  readiness,  and 
information  management.  Technical  competencies  particularly  important  to  the  S3/XO  include 
leveraging  digital  C2  technology  to  aid  in  situational  awareness  and  understanding  and 
information  dominance  (Leibrecht,  Lockaby,  Perrault,  &  Meliza,  2004).  Lacking  technical 
competence  in  these  areas  has  direct  and  indirect  implications  for  tactical  competence.  Directly, 
technical  competence  enables  the  S3/XO  to  resource,  allocate,  exploit,  and  apply  available 
digital  C2  equipment  for  enhancing  mission  planning  and  tracking  the  progress  of  operations. 
Indirectly,  technical  competence  enables  the  S3/XO  to  train  the  staff  and  subordinate  officers  on 
the  digital  skills  required  to  act  decisively  in  the  area  of  operations. 

Individual  technical  competence  is  particularly  challenging  to  develop  for  multiple 
reasons,  not  the  least  of  which  are  limited  availability  of  systems  on  which  to  train,  limited 
access  to  alternative  digital  training  techniques,  rapid  rate  of  change  in  the  capability  of  digital 
C2  systems,  and  assignment  patterns  that  remove  officers  from  operational  units  for  several  years 
at  a  time.  Collective  technical  competence  as  developed  by  the  S3/XO  must  address  such  factors 
as  bandwidth  constraints  on  what  information  can  be  shared  when,  stovepiped  digital  systems  for 
each  BOS  (where  access  to  digital  systems  is  equivalent  across  BOSs,  which  is  not  always  the 
case),  differing  versions  of  the  same  digital  systems  at  different  echelons,  and  asynchronous 
system  upgrades  across  BOSs  (Leibrecht  et  ah,  2004). 

Supporting  the  S3/XO  in  developing  individual  technical  competence  involves  sharing 
knowledge  as  to  what  training  is  available  for  using  the  digital  C2  systems  that  each  staff  officer 
must  use  to  conduct  planning  and  track  operations,  what  tactics,  techniques,  and  procedures 
(TTPs)  have  been  used  to  leverage  the  capability  of  these  systems,  and  what  resources  are 
available  for  better  understanding  the  latest  status  of  these  systems  and  how  they  can  be  used  to 
achieve  information  dominance.  Supporting  the  S3/XO  in  developing  collective  technical 
competence  in  their  units  involves  sharing  digital  SOPs,  TTPs  for  addressing  bandwidth  and 
access  constraints,  and  digital  skills  training  techniques  used  in  unit  drills,  among  others. 

Conceptual  Competence.  Conceptual  competence  is  comprised  of  establishing  with 
others  one’s  intent,  taking  a  systems  perspective  to  understanding  situations,  reasoning  critically, 
thinking  creatively,  thinking  reflectively,  adapting  to  changing  circumstances,  and  predicting  the 
second  and  third  order  effects  of  one’s  own  actions  on  the  environment.  By  the  time  officers 
become  majors,  they  are  expected,  as  a  matter  of  course,  to  take  initiative  to  identify  and  solve 
problems  and  learn  from  their  experiences.  They  must  develop  novel  solutions  to  difficult 
problems  in  an  operational  environment  where  the  enemy  is  asymmetric  and  constantly  modifies 
his  tactics.  They  must  respond  quickly  and  adaptively  to  environmental  changes,  anticipating 
change  where  they  can  and  thinking  flexibly  in  unanticipated  circumstances.  Finally,  they  must 
empower  their  fellow  officers  to  take  their  own  initiative  by  making  their  intentions  clear  and 
developing  conceptual  competence  in  others.  Conceptual  competence  enables  S3s/XOs  to 
oversee  effectively  the  administrative  and  logistical  support  and  training  of  their  unit  at  a  time 
when  resources  are  extremely  tight  and  the  structure  of  the  unit  itself  is  changing.  Conceptual 


77 


competence,  through  increased  adaptability  and  innovation,  also  enables  S3s/XOs  to  better 
support  mission  planning  and  staff  integration  at  a  time  when  many  aspeets  of  tacties  are 
uncertain  and  the  exact  role  of  specific  staff  officers  in  supporting  the  planning  process  is 
uncertain. 

The  increased  demand  in  the  Army  for  such  competencies  as  critical  reasoning,  ereative 
thinking,  and  adaptive  ability  has  not  yet  been  accompanied  by  the  cultural  change  necessary  to 
allow  junior  officers  the  opportunity  to  develop  these  eompeteneies  (Wong,  2002).  Emphasis  has 
been  placed  in  the  schoolhouse  on  developing  adaptive,  innovative  leaders  and  eritieal  thinkers 
(e.g.,  Fiseher,  Spiker,  &  Reidel,  2000;  Lussier  &  Shadrick,  2004),  but  lacking  the  opportunity  to 
exercise  them  in  the  field,  officers  cannot  retain  new  eompeteneies  developed  in  the 
schoolhouse.  Although  the  war  in  Iraq  has  shifted  tactical  emphasis  down  to  the  company  and 
platoon  levels,  providing  junior  officers  unprecedented  experienee  with  innovative  eommand 
decision-making,  most  current  S3/XOs  have  assumed  their  duty  position  without  having  had  this 
experience  (Wong,  2004).  As  a  result,  many  S3s/XOs  must  use  their  immediate  experiences  to 
gain  conceptual  competence. 

Supporting  the  S3/XO  in  rapidly  gaining  conceptual  competenee  from  his  immediate 
experienees  must  involve  promoting  the  S3/XO’s  inclination  to  recognize  and  use  his 
experiences  and  those  of  others  as  learning  opportunities.  This  requires  providing  an 
environment  in  which  the  S3/XO  (a)  is  encouraged  to  make  explicit  the  situations  in  which  he 
has  worked,  the  problems  he  has  identified,  the  solutions  he  has  created,  and  their  corresponding 
outcomes;  (b)  can  safely  discuss  a  problem  he  is  facing  with  others  who  have  faced  similar 
problems  in  order  to  clarify  his  definition  of  the  problem,  identify  a  range  of  possible  solutions, 
or  seek  feedback  on  his  proposed  solution;  and  (c)  can  rapidly  access  best  praetiee  in  addressing 
problems  similar  to  that  which  he  is  facing.  Such  an  environment  would  facilitate 
resourcefulness,  situated  reflection,  and  learning  orientation,  which  serve  as  a  basis  for 
conceptual  competence  (Cianciolo  et  ah,  2004). 

Interpersonal  Competence.  Aecording  to  Horey,  Fallesen,  Morath,  Cronin,  et  al.  (2004), 
eight  interpersonal  competencies  characterize  the  future  Army  leader.  Three  that  have  not 
already  been  addressed  in  previous  sections  are  (1)  vitalizing  a  positive  climate;  (2)  arming  self 
to  lead;  and  (3)  extending  personal  influence.  Vitalizing  a  positive  climate  involves  setting  the 
conditions  for  positive  attitudes  and  effective  work  behaviors.  Arming  self  to  lead  is  that  activity 
the  leader  engages  in  to  ensure  he  is  prepared  for  challenges,  aware  of  himself  and  his 
environment,  and  engaged  in  lifelong  learning.  Extending  influence  involves  ensuring  that  one’s 
influence  goes  beyond  the  direct  lines  of  authority  and  chains  of  command.  For  the  S3/XO, 
interpersonal  eompetence  means  creating  a  climate  in  which  staff  actions  are  cohesive  and 
integrated  (Olmstead,  1 992),  where  each  staff  officer  knows  and  is  empowered  to  perform  his 
role  on  the  team,  where  personality  differences  do  not  affect  the  functioning  of  the  staff,  and 
where  each  staff  officer’s  effort  is  focused  on  collectively  supporting  the  commander. 
Interpersonal  competence  also  means  the  S3/XO  must  take  the  initiative  to  build  a  constructive 
relationship  with  the  commander,  fellow  staff  officers  (including  those  at  higher  or  lower 
echelons),  the  command  sergeant  major,  and  subordinates.  These  relationships  must  enable  him 
to  determine  what  is  expected  of  him  to  lead  the  staff  effectively,  to  identify  his  strengths  and 
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weaknesses  as  they  relate  to  leading  in  his  current  situation,  and  to  understand  the  particular 
challenges  to  the  effectiveness  of  his  staff  and  unit. 

The  S3/XO’s  development  or  display  of  interpersonal  competence  can  be  difficult  in 
units  where  the  command  climate  is  negative  or  apathetic,  where  the  previous  S3/XO  has  not  left 
behind  information  useful  for  functioning  within  the  unit,  where  there  is  a  great  deal  of  turnover 
of  staff  officers,  where  there  are  personnel  shortages  that  reduce  time  available  or  staff 
capability,  and  where  staff  officers  are  inexperienced  and  unlikely  to  take  initiative.  Some 
combination  of  these  conditions  characterizes  most  units,  especially  high  turnover  rates, 
personnel  shortages,  and  inexperienced  staff  officers. 

Supporting  the  S3/XO  in  developing  and/or  demonstrating  interpersonal  competence 
requires  that  S3s/XOs  have  access  to  the  officers  who  preceded  them— to  be  able  to  discuss  the 
peculiarities  of  the  unit  to  which  they  have  been  assigned  and  to  share  methods  for  working 
effectively  in  the  unit.  S3s/XOs  must  also  be  exposed  to  opportunities  to  learn  general  methods 
for  dealing  with  turnover  (e.g.,  SOPs  dealing  with  collective  staff  tasks,  such  as  planning)  and 
inexperienced  staff  officers  (e.g.,  exercises  or  other  techniques  to  foster  initiative).  Finally,  the 
S3/XO  must  be  aware  of  what  worked  or  didn’t  work  for  others  who  dealt  with  difficult 
interpersonal  situations,  such  as  low  morale,  poor  command  climate,  or  times  of  high  stress 
and/or  workload.  Storytelling  is  often  an  effective  vehicle  for  sharing  tacit  knowledge  about  how 
to  handle  challenging  interpersonal  issues  (Cianciolo,  Matthew,  Wagner,  &  Sternberg,  in  press). 

Actions  -  Doing 

FM  22-100  (DOA,  1999)  outlines  three  broad  categories  of  action  in  which  successful 
field  grade  officers  must  demonstrate  competence — influencing,  operating,  and  improving. 
Influencing  involves  communicating  effectively,  leading  by  example,  and  moving  beyond 
reactive  activity  to  engage  in  anticipatory  decision  making.  Operating  involves  effective  mission 
planning,  preparation,  and  execution,  as  well  as  objective  information  assessment.  Improving 
involves  engaging  in  self-development  and  lifelong  learning,  building  successful  teams,  and 
promoting  self-awareness.  Several  aspects  of  these  competencies,  such  as  synchronization, 
building  staff  integration  and  cohesion,  and  preparing  oneself  to  lead,  have  been  addressed 
previously  and  will  not  be  addressed  here.  Only  unique  elements  are  discussed. 

Influencing.  Influencing  involves  exemplifying  Army  values  and  “warrior  ethos,” 
communicating  effectively,  and  making  effective  decisions,  all  of  which  have  been  addressed  in 
the  preceding  discussion. 

Operating.  In  addition  to  synchronization  during  mission  plarming  and  execution, 
operating  involves  the  gathering  and  analysis  of  information  in  order  to  understand  its  reliability, 
validity,  and  sources.  It  is  important  that  information  assessment  be  as  unbiased  and  objective  as 
possible.  Both  the  S3  and  XO  must  be  able  to  make  actionable  recommendations  to  the 
commander  on  the  basis  of  sound  information  about  the  threat  and  the  environment. 
Recommendations  must  also  be  made  on  the  basis  of  an  accurate  understanding  of  the  friendly 
situation. 
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The  assessment  of  information  on  the  threat  and  the  environment  is  particularly 
challenging  in  today’s  operational  environment  because  there  is  a  great  deal  more  information 
being  gathered  by  a  greater  diversity  in  sensory  capability.  This  capability  includes  advanced 
sensing  technology  as  well  as  the  use  of  Soldiers  and  informants  in  the  civilian  populace  as 
human  intelligence  gatherers.  The  primary  threats  to  the  reliability  and  validity  of  incoming 
information  about  the  threat  and  the  environment  include  (a)  lack  of  information  integration 
necessary  to  determine  the  true  meaning  and  implications  of  intelligence  data;  and  (b) 
uncertainty  regarding  the  credibility  of  civilian  informants.  The  primary  threats  to  the  reliability 
and  validity  of  incoming  information  about  the  friendly  situation  include  (a)  incompleteness  of 
information  due  to  inadequate  awareness  of  those  reporting  the  information;  and  (b) 
misrepresentation  of  information  (e.g.,  troop  readiness  status)  due  to  lack  of  awareness  or 
reporting  of  future  projections  rather  than  current  status. 

Discussion  and  knowledge  sharing  that  supports  the  S3/XO  in  effectively  assessing 
information  should  cover  such  topics  as  methods  for  integrating  information  coming  from 
diverse  sources,  techniques  for  determining  the  credibility  of  civilian  informants,  and  methods 
for  ensuring  that  staff  estimates  and  other  information  regarding  friendly  status  are  complete 
and/or  accurate. 

Improving.  Improving  involves  developing  a  lifelong  learning  orientation,  empowering 
subordinates  to  take  on  increased  responsibility  and  initiative,  and  building  cohesive  teams,  all  of 
which  have  been  addressed  in  the  preceding  discussion. 

Member  Competence  and  Professionalism  -  A  Proposed  Framework  for  Assessment 

To  assess  the  impact  that  S3-XO.net  could  potentially  have  on  member  competence  and 
professionalism,  a  red-amber-green  rating  system  analogous  to  the  one  used  earlier  in  this  report 
was  used.  The  red-amber-green  criteria  for  each  aspect  of  member  competence  and 
professionalism  are  presented  in  the  tables  below.  It  is  important  to  note  that  the  below 
assessment  criteria  are  recommended  as  a  starting  point  and  that  several  of  the  assessment 
criteria  presented  here  must  change  as  current  issues  evolve.  In  addition,  these  criteria  may  not 
be  exhaustive  or  reflect  the  areas  of  greatest  joint  interest  to  the  Army  and  S3-XO.net  members. 
Finally,  it  should  be  noted  that  the  proposed  framework  is  for  evaluating  potential  impact  of  the 
forum,  rather  than  assessing  impact  directly.  The  assumption  behind  this  approach  is  that  (a)  it  is 
more  feasible  and  more  tractable  to  assess  the  potential  for  impact  rather  than  impact  directly; 
and  (b)  if  direct  impact  is  of  particular  interest,  the  proposed  approach  is  useful  for  determining 
whether  taking  on  such  an  effort  would  provide  useful  additional  information. 

Impact  on  Member  Competence  and  Professionalism  -  Assessment  o/S3-XO.net 

The  impact  of  S3-XO.net  on  the  competence  and  professionalism  of  its  members  was  not 
fonnally  assessed  in  the  present  investigation  because  status  ratings  of  conversation,  content,  and 
connections  were  all  red-amber.  More  specifically,  conversation  activity  and  quality  both  were 
assigned  a  red-amber  status,  participating  members  and  facilitating  connections  among  members 
were  assigned  a  red  status,  and  identifying  experts  was  assigned  an  amber  status.  This  suggests 
that  what  conversation  was  occurring  was  not  of  good  quality  and  may  likely  not  have  been 
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among  individuals  of  the  target  membership.  Given  these  status  ratings,  it  is  unlikely  that  S3- 
XO.net  could  have  a  potential  impact  status  greater  than  red  or  red-amber. 

Formal  analysis  of  impact  can  be  expected  to  provide  information  above  and  beyond 
analysis  of  social  and  intellectual  capital  when  status  ratings  of  conversation,  content,  and 
connections  are  amber-green  or  green.  Impact  analysis  at  that  point  addresses  whether  the  social 
and  intellectual  capital  actively  developed  in  the  forum  is  in  line  with  joint  organizational/forum 
member  needs.  Direct  assessment  of  impact  addresses  whether  relevant  social  and  intellectual 
capital  development  can  overcome  the  cultural,  logistical,  administrative,  or  other  challenges  to 
effective  individual  performance. 

Informal  assessment  of  S3-XO.net  indicates  that  several  of  the  topics  identified  as 
relevant  to  having  an  impact  on  officer  competence  and  professionalism  are  present  in  the  forum, 
including,  for  example,  leadership  challenges  in  transformation,  combined  arms  training  in  a  unit 
of  action,  the  military  decision-making  process  in  a  collaborative  unit,  and  so  on.  However, 
conversation  tended  to  be  quite  sparse  and  stories  were  infrequently  used  to  share  tacit 
knowledge. 


Impact  on  Member  Competence  and  Professionalism  -  Recommendations 

Our  general-level  recommendation  for  ensuring  that  conversation  in  the  forum  is  relevant 
to  joint  Army  and  S3-XO.net  member  needs  is  to  survey  the  available  literature  addressing  issues 
currently  challenging  the  organization,  particularly  as  it  relates  to  S3s/XOs,  and  to  interview 
individuals  who  have  keen  insight  into  the  issues  facing  S3s/XOs  and  the  Army  as  a  whole. 
Ideally  facilitators  in  the  forum  are  members  of  the  professional  community  and  have  an  organic 
sense  of  the  issues,  but  in  the  case  that  they  do  not,  it  is  essential  to  know  what  or  who  to  target 
to  generate  useful  information  in  the  forum.  Non-expert  facilitators  should  have  an  expert 
advisor  who  helps  them  identify,  anticipate,  and  address  member  needs. 

Examples  of  literature  or  other  documentation  that  may  be  helpful  in  identifying  S3/XO 
needs  include  professional  articles  found  in  Military  Review,  which  is  targeted  toward  field  grade 
officers,  and  CALL  reports  featuring  combat  training  center  trends,  particularly  as  they  relate  to 
the  battle  staff.  Many  of  the  National  Training  Center  assessment  teams  (e.g..  Cobras,  Broncos, 
etc.)  have  forums  in  AKO  that  post  trends.  These  may  be  helpful  resources.  Occasionally, 
branch-specific  professional  publications,  such  as  Armor  magazine  or  Field  Artillery  magazine, 
will  have  articles  addressing  battalion  staff  issues.  Up  to  date  information  about  organizational 
initiatives  released  in  the  Army  News  or  posted  on  organizational  websites  may  provide  useful 
information  for  anticipating  S3/XO  needs  and  prioritizing  them  for  addressing  in  the  forum 
(Captain  Robert  Thornton,  personal  communieation).  Parameters  and  documents  produced  by 
the  Strategic  Studies  Institute  at  the  U.S.  Army  War  College  provide  insight  into  issues 
challenging  the  Army  as  an  organization. 

Fundamentally,  the  most  important  people  to  interview  about  what  conversation  and 
content  on  the  forum  will  be  most  helpful  to  current  and  future  S3s/XOs  are  current  and  future 
S3s/XOs.  It  can  be  quite  difficult  to  win  time  from  these  individuals,  especially  current  S3s/XOs, 
but  of  the  people  we  interviewed,  nearly  all  of  them  were  enthusiastic  about  horizontal 
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Table  20.  A  Proposed  Framework  for  Assessing  Potential  Impact  on  Officer  Values,  Personal  Characteristics, 
_ and  Professionalism  (Be) 
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knowledge  sharing  with  peers.  They  were  generous  with  their  time  and  provided  very  helpful 
insights.  Additional  people  to  approach  about  identifying  needs  include  the  authors  of  relevant 
articles  found  in  the  professional  literature.  Through  their  writing,  these  individuals  have 
demonstrated  their  expertise  and  interest  in  particular  topics,  and  willingness  to  spend  time  to 
bring  these  topics  to  the  attention  to  others.  These  authors  typically  have  a  wealth  of  experience 
to  draw  on  and  easily  can  be  located  through  AKO.  Other  insightful  people  include  the 
observer/controllers  at  the  Combat  Training  Centers  and  senior  members  of  the  community. 

Locating  individuals  to  discuss  the  needs  of  S3s/XOs  simultaneously  helps  facilitators 
develop  social  capital,  people  they  can  rely  on  to  answer  a  related  question  that  comes  up  in  the 
forum  or  to  recommend  individuals  who  have  special  insight  into  that  question.  The  development 
of  this  social  capital  is  especially  important  when  facilitators  are  themselves  not  current  or  future 
S3s/XOs.  In  addition,  if  interaction  between  facilitators  and  interviewees  is  constructive,  positive 
word  about  the  forum  is  spread  even  though  the  activity  was  not  part  of  a  formal  advertising 
campaign. 


S3-XO.net  Impact  on  Unit  Effectiveness 

As  with  managing  the  scope  of  individual-level  impact  assessment,  it  is  important  to 
manage  the  scope  of  unit-level  impact  assessment  by  targeting  those  aspects  of  unit  effectiveness 
for  which  common  or  persistent  problems  have  been  identified.  In  addition,  for  a  fair  assessment 
of  impact,  “unit”  must  be  defined  in  such  a  way  that  unit  effectiveness  can  be  directly  influenced 
by  the  target  membership  of  the  APF  whose  impact  is  being  assessed.  Because  the  target 
membership  of  S3-XO.net  is  comprised  of  current  and  future  S3s  and  XOs  at  the  BN  and  BDE 
level,  “unit”  is  here  defined  as  the  BN  or  BDE  staff  Improvements  in  unit-level  effectiveness 
should  therefore  address  common  or  persistent  problems  identified  in  BN  or  BDE  staff 
performance. 

The  same  feasibility  issues  surrounding  the  assessment  of  direct  impact  on  individual- 
level  competence  and  professionalism  also  apply  to  using  the  quasi-experimental  approach  to 
assess  directly  unit-level  impact.  Here  too,  a  reasonable  proxy  is  to  assess  whether  or  not  the 
content  and  discussion  in  S3-XO.net  addresses  the  key  problem  areas  in  BN/BDE  staff 
effectiveness.  The  social  and  intellectual  capital  assessment  criteria  as  applied  in  the  proposed 
framework  for  evaluating  potential  impact  on  individual-level  effectiveness  could  be  used  to 
assess  the  potential  impact  of  S3-XO.net  on  unit-level  effectiveness  in  the  targeted  areas  of 
interest. 

The  following  section  presents  an  overview  of  the  role  of  the  S3/XO  on  the  BN/BDE 
staff,  highlighting  general  areas  where  difficulty  has  been  identified'^.  We  used  a  combination  of 
literature  review  and  interviews  with  subject  matter  experts  to  understand  the  role  of  the  S3/XO 


The  present  overview  is  focused  primarily  on  problems  occurring  during  mission  plaiming  and  execution.  This 
focus  reflects  that  of  the  available  literature  on  staff  performance,  which  comes  from  combat  training  center 
assessments  and  descriptions  of  combat  lessons  learned.  A  survey  of  the  non-mission  challenges  faced  by  S3s/XOs 
and  their  implications  for  unit  effectiveness  would  provide  valuable  information  for  evaluating  the  impact  of  S3- 
XO.net.  However,  the  difficulty  obtaining  an  understanding  of  non-mission  challenges  suggests  that  such  challenges 
may  be  of  less  interest  to  organizational  decision  makers  than  mission-related  challenges. 
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and  to  identify  areas  of  difficulty.  A  list  of  the  subject  matter  experts  we  interviewed  is  provided 
in  Appendix  A.  If  S3-XO.net  is  to  have  an  impact  on  unit  effectiveness  and,  by  extension, 
organizational  effectiveness,  these  areas  should  represent  a  significant  proportion  of  the 
productive  discussion  and  knowledge  sharing  occurring  in  the  forum. 

Requirements  of  the  BN/BDE  Staff 

Ultimately,  the  staff  must  serve  as  an  extension  of  the  commander  by  being  his  eyes,  ears, 
and  voice  in  all  matters  of  command,  whether  in  garrison  or  theater  [U.S.  Army  Infantry  School 
(USAIS),  1937].  The  purpose  of  the  staff  is  to  make  the  commander’s  authority  known  down  the 
chain  of  command  and  to  inform  the  commander’s  decision  making  through  the  supply  of 
timely,  relevant  information  (USAIS,  1937).  Whereas  the  commander  leads  the  unit,  the  staff 
manages  the  day-to-day  activity  of  the  unit,  taking  advantage  of  every  opportunity  to  increase  the 
efficiency  of  staff  functioning  (FM  6-0;  DOA,  2003). 

The  XO  is  the  leader  of  the  staff,  normally  delegated  executive  management  authority  by 
the  commander.  He  directs  staff  tasks,  oversees  staff  coordination,  and  ensures  efficient  and 
prompt  staff  actions  (FM  6-0).  All  coordinating  staff  officers  (e.g.,  intelligence  officer, 
operations  officer,  and  logistics  officer),  special  staff  officers  (e.g.,  engineering,  fire  support, 
etc.),  and  attachments  are  directly  responsible  to  the  XO  (FM  6-0).  He  is  responsible  for  staff 
training  and  subordinate  unit  readiness,  and  ensures  that  the  staff  functions  as  a  collective  and 
that  their  efforts  are  maximally  coordinated  with  the  commander’s  intent.  He  also  maintains 
infonnation  flow  between  the  commander  and  the  rest  of  the  staff.  The  XO  must  establish  a  close 
relationship  with  the  commander,  understanding  how  the  commander’s  personal  characteristics 
and  command  style  influence  his  intent  (FM  6-0). 

The  S3  is  responsible  for  coordinating  and  overseeing  unit  training,  operations  and  plans, 
and  force  development  and  modernization  (FM  6-0).  He  must  ensure  that  operations  are 
conducted  according  to  the  commander’s  intent,  which  includes  troop  preparation,  mission 
planning,  and  coordination  of  execution,  among  other  responsibilities.  Together  with  the 
commander  he  develops  the  mission  essential  task  list,  then  he  identifies  the  associated  training 
requirements  and  conducts  training  to  ensure  that  subordinate  units  meet  these  requirements. 
During  operations,  the  S3  must  work  closely  with  the  other  coordinating  and  special  staff 
officers  in  order  to  ensure  that  tactical  operations  are  adequately  resourced  and  synchronized.  He 
fosters  the  development  and  modernization  of  the  unit  by  recommending  and  maintaining  a  force 
structure  that  optimally  balances  resource  allocation  with  mission  requirements. 

Key  Challenges  -  The  Entire  Staff 

Advise  and  Inform  the  Commander.  In  order  to  effectively  advise  and  inform  the 
commander,  the  staff  must  “provide  relevant  information  to  inform  the  commander’s  situational 
awareness/understanding  and  decision  making”  (FM  5-0;  DOA,  2002).  Staffs  fairly  reliably  have 
difficulty  providing  an  effective  analysis  of  incoming  information  such  that  the  implications  or 
the  “so  what?”  of  the  information  for  decision-making  are  clear  (e.g..  White,  2001).  This 
difficulty  arises  particularly  during  staff  planning  (e.g.,  mission  analysis)  but  also  during  mission 
execution.  For  example,  recent  observations  from  the  National  Training  Center  indicate  that 
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staffs  often  fail  to  create  a  decision  support  template  (DST)  or  decision  support  matrix  (DSM) 
during  mission  planning.  The  DST/DSM  specifies  the  “so  what?”  of  information  by  linking  the 
commander’s  critical  information  requirements  (CCIR)  to  points  in  time  and  space  on  the 
battlefield  where  intelligence  must  be  gathered  and  a  decision  must  be  made.  Moreover,  staffs 
who  do  create  a  DST/DSM  often  fail  to  refer  to  it  in  order  to  filter  incoming  information  and 
inform  the  commander  in  a  timely  manner  that  a  decision  is  at  hand. 

Difficulty  analyzing  incoming  data  for  their  implications  may  stem  from  a  variety  of 
causes.  Among  the  most  common  of  these  causes  are  ineffective  battle  rhythm  in  the  tactical 
operations  center  (TOC)  leading  to  fatigue  and  error,  absent  or  ignored  TOC  SOPs  for  managing 
information,  and  failure  to  understand  the  commander’s  intent.  Shortfalls  in  these  areas  are 
particularly  problematic  in  asymmetric  warfare,  where  information  dominance  is  a  critical  source 
of  combat  advantage  (Kasales,  2002).  The  S3  and  XO  are  both  directly  responsible  for  ensuring 
that  information  management  during  planning  and  execution  runs  smoothly  and  supports  the 
commander’s  decision  making. 

Prepare  Plans  and  Orders.  The  preparation  of  plans  and  orders  involves  executing  the 
military  decision-making  process  (MDMP).  Staffs  have  had  difficulty  executing  the  MDMP  for 
decades.  These  difficulties  include  managing  time  during  the  planning  process,  conducting 
effective  mission  analysis  (e.g.,  identifying  enemy  decision  points),  fully  developing  a  course(s) 
of  action,  conducting  an  effective  war-game,  and  producing  a  synchronized  plan  (e.g.,  CALL, 
1996,  1997,  1998,  2003).  The  challenge  to  effeetively  apply  the  MDMP  has  only  increased  as 
units  face  an  asymmetric  enemy.  Specifically,  the  latest  trends  from  the  National  Training  Center 
indicate  that  staffs  treat  new  methods  for  targeting  operational  efforts  (e.g.,  effects-based 
operations)  as  separate  from  the  MDMP  and  therefore  discard  the  MDMP.  Limitations  in 
understanding  how  the  MDMP  applies  in  conventional  warfare  are  preventing  the  application  of 
the  MDMP  to  planning  counterinsurgency  operations  and  stability  and  support  operations.  In 
addition,  staffs  often  take  too  much  time  to  plan,  failing  to  allow  subordinate  units  enough  time 
to  plan  their  own  missions. 

Some  causes  of  difficulty  in  applying  the  MDMP  include  ineffective  battle  rhythm  in  the 
TOC  leading  to  fatigue  and  error,  inadequate  staff  training,  unfamiliarity  with  doctrine,  poor 
integration  or  cohesion  within  the  staff,  absent  or  ineffective  planning  SOPs,  and  failure  to 
understand  the  commander’s  intent  (and  to  seek  guidance  when  his  intent  is  vague).  The  XO  is  in 
charge  of  training  and  coordinating  the  staff  (FM  6-0;  DOA,  2003).  He  must  set  a  planning 
timeline  and  ensure  that  the  timeline  is  adhered  to  (Pleban,  Thompson,  «&  Valentine,  1993).  He 
must  also  oversee  the  planning  process,  ensuring  that  his  planning  SOPs  are  put  into  action  and 
that  staff  efforts  are  integrated  and  aligned  with  the  commander’s  intent.  This  requires  a  close 
relationship  with  the  commander  and  proactive  personnel  management,  with  particular  attention 
to  the  tendency  for  certain  staff  officers  (e.g.,  the  S4,  forward  support  battalion  support 
operations  officer)  to  be  excluded  from  the  planning  process  (Wilson  &  Snow,  2003).  The  S3 
must  take  a  systems  approach  to  planning  and  execution,  visualizing  and  understanding  the 
importance  of  the  contribution  of  each  BOS  to  achieving  decisive  effects. 

Manage  Information  Within  Own  Field  of  Interest.  Information  management  is  an 
especially  difficult  task  for  battle  staffs,  particularly  as  it  relates  to  understanding  the  CCIR  and 
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their  implications  for  developing  the  intelligence  collection  plan,  analyzing  incoming 
information  to  determine  its  relevance  to  the  CCIR  and  the  commander’s  decision  making,  and 
disseminating  information  in  order  to  answer  the  CCIR.  This  task  becomes  even  more 
challenging  during  stability  and  support  operations  conducted  within  the  province  of  an 
asymmetric  enemy  because  intelligence  collection  operations  involve  everyone  in  the  unit  (i.e., 
as  opposed  to  involving  only  a  scout  platoon  or  brigade  military  intelligence  assets).  This 
increase  in  number  of  Soldiers  collecting  information,  combined  with  digital  situational 
awareness  technology,  dramatically  increases  the  amount  of  information  entering  the  TOC.  In 
order  to  sift  through  large  amounts  of  incoming  information,  staff  officers  must  know  and 
communicate  what  they  are  looking  for,  who  they  expect  to  provide  the  information,  and  the 
time  by  which  they  must  have  the  information  they  seek. 

Some  possible  causes  for  difficulty  with  managing  information  include  failure  to 
understand  the  commander’s  intent,  lack  of  planning  and/or  TOC  SOPs,  lack  of  integration 
among  the  S3  and  the  intelligence  officer,  and  fatigue  or  lack  of  time  available  in  the  TOC  (due 
to  ineffective  time  management  or  inadequate  battle  rhythm).  As  stated  previously,  the  S3  and 
XO  are  both  directly  responsible  for  ensuring  that  information  management  during  planning  and 
execution  runs  smoothly  and  supports  the  commander’s  decision  making.  The  XO  must  lead 
information-management  by  making  clear  the  commander’s  intent  and  initial  information  needs, 
ensuring  that  the  staff  thoroughly  identifies  the  commander’s  additional  information  needs,  and 
establishing/implementing  the  SOPs  for  managing  information  in  the  TOC  during  mission 
execution.  The  S3  aids  in  the  information  analysis  effort  by  effectively  representing  the 
commander’s  concept  of  operations  and  coordinating  with  the  intelligence  officer  during  mission 
planning. 

Perform  Staff  Coordination.  Staff  integration  has  eluded  battle  staffs  for  decades  (e.g., 
Grossman,  1994;  Olmstead,  1992).  That  is,  staff  officers  generally  do  not  share  information  with 
one  another  during  the  planning  process,  do  not  understand  and  anticipate  each  others’  needs, 
and  do  not  coordinate  with  one  another  when  developing  components  of  the  mission  plan.  The 
lack  of  integration  among  staff  officers  has  deleterious  effects  on  the  quality  of  planning 
products— not  only  the  plan  itself,  but  also  such  products  as  the  synchronization  matrix  and 
decision  support  matrix.  A  plan  that  is  not  synchronized  will  likely  be  ineffective,  but  could  also 
be  dangerous,  for  example  through  increased  risk  of  fratricide.  In  addition,  difficulty  with  staff 
integration  not  only  occurs  within  the  staff  of  a  particular  unit,  but  also  between  the  staffs  of 
superior  and  subordinate  (e.g.,  BDE  and  BN)  units. 

There  are  several  potential  causes  of  inadequate  staff  integration,  including  a  rapid  rate  of 
turnover  within  the  staff  team,  lack  of  clarity  regarding  the  commander’s  intent  for  individual 
staff  roles  (e.g.,  XO  vs.  S3),  lack  of  awareness  regarding  the  doctrinal  roles  and  information 
requirements  of  each  staff  officer  (including  oneself,  Cianciolo  &  Sanders,  to  appear), 
underestimation  of  the  importance  of  each  staff  officer,  and  interpersonal  conflict. 

Challenges  -  XO 

The  following  duties/responsibilities  (FM  6-0;  DOA,  2003)  are  especially  challenging  for 
the  XO: 
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•  Integrate  and  synchronize  plans  and  orders 

•  Supervise  management  of  the  CCIR 

•  Establish,  manage,  and  enforce  the  staff  planning  timeline 

•  Conducting  staff  training 

•  Ensuring  staff  work  conforms  to  the  mission,  commander’s  guidance,  and  time  available 

•  Ensuring  the  staff  integrates  and  coordinates  its  activities  internally  and  with  higher, 
subordinate,  supporting,  supported,  and  adjacent  commands 

•  Ensuring  all  staff  sections  participate  in  and  provide  functional  expertise  to  intelligence 
preparation  of  the  battlefield 

•  Directing  and  supervising  staff  planning 

•  Supervising  intelligence,  surveillance,  and  reconnaissance  (ISR)  integration 

There  are  several  factors  that  create  difficulty  in  accomplishing  the  above  listed 
duties/responsibilities,  including  turnover  within  the  staff,  inexperienced  staff  officers  who  lack 
initiative,  ineffective  time  management  by  the  next  higher  staff,  distractions  during  the  planning 
process  (e.g.,  the  XO  is  pulled  away  from  the  planning  process  to  handle  a  logistics  problem), 
tactical  ambiguity  arising  from  significant  unit  reorganization,  interpersonal  conflict  within  the 
staff,  and  environmental  and  cultural  pressures  to  stovepipe  staff  officer  knowledge  and  activity. 
Through  effective  leadership  and  the  use  of  such  tools  as  SOPs,  graphical  timelines,  and  staff 
training  TTPs,  the  XO  is  expected  to  deal  with  these  factors  to  run  an  effective  staff.  Because  the 
XO  is  the  leader  of  the  staff,  his  effectiveness  translates  into  the  effectiveness  of  the  staff  as  a 
whole.  Shortfalls  in  accomplishing  the  duties/responsibilities  listed  above  readily  can  be  seen  in 
the  challenges  faced  by  the  staff  as  a  whole  described  in  the  previous  section. 

Challenges  -  S3 

The  following  duties/responsibilities  (FM  6-0,  DOA,  2003)  are  especially  challenging  for 

the  S3: 

Training 

•  Conducting  training  within  the  command 

•  Preparing  training  guidance  for  the  commander’s  approval 

•  Helping  the  commander  develop  the  unit’s  mission  essential  task  list  (METL) 

•  Identifying  training  requirements,  based  on  the  unit  METL  and  training  status 

•  Determining  requirements  for  and  allocation  of  training  resources 

Plans  and  Operations 

•  Synchronizing  tactical  operations  with  all  staff  sections 

•  Reviewing  plans  and  orders  for  synchronization  and  completeness 

•  Ensuring  necessary  combat  support  requirements  are  provided  when  and  where  required 

•  Integrating  ISR  into  the  concept  of  operations 

•  Integrating  and  managing  the  ISR  effort  through  an  integrated  staff  process  and 
procedure 

•  Developing  the  ISR  plan  (with  the  rest  of  the  staff) 
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•  Synchronizing  ISR  with  the  overall  operation  throughout  the  operations  process  (with  the 
rest  of  the  staff). 

•  Integrating  fire  support  into  operations 

•  Determining  combat  service  support  resource  requirements  (with  the  personnel  and 
logistics  officer). 

•  Participating  in  course  of  action  and  decision  support  template  development  (with  the 
intelligence  officer  and  fire  support  coordinator). 

Force  Development  and  Modernization 

•  Processing  procedures  for  unit  activation,  inactivation,  establishment,  discontinuance, 
and  reorganization  (force  accounting) 

•  Evaluating  the  organizational  structure,  functions,  and  workload  of  military  and  civilian 
personnel  to  ensure  their  proper  use  and  requirements  (manpower  utilization  and 
requirements) 

•  Developing  and  revising  unit  force  data  for  documenting  any  changes  to  the  modification 
table  of  organization  and  equipment  (MTOE)  and  modification  table  of  distribution  and 
allowances  (MTDA) 

•  Conducting  formal,  on-site  manpower  and  equipment  surveys 

•  Ensuring  MTOE  and  MTDA  documents  reflect  the  minimum  essential  and  most 
economical  equipment  needed  for  the  assigned  mission 

•  Determining  qualitative  and  quantitative  personnel  requirements  for  new  equipment  and 
systems 

Unit  training  is  a  major  challenge  for  the  S3,  due  to  equipment  shortages  to  conduct 
training,  shortages  of  NCOs  to  assist  in  training,  and  lack  of  doctrinal  guidance  to  deal  with  the 
training  implications  of  force  reorganization  and  digitization.  The  major  challenge  to  conducting 
plans  and  operations  for  the  S3  is  coordinating  with  the  other  staff  officers  in  order  to  develop 
and  execute  a  unified  plan.  The  importance  of  combat  service  and  combat  service  support  to 
mission  success  tends  to  be  underestimated,  and  as  a  result  several  coordinating  and  special  staff 
officers  are  left  out  of  the  planning  process.  During  mission  execution,  when  the  main  command 
post  or  TOC  and  the  combat  trains  command  post  are  geographically  distributed,  staff  integration 
becomes  much  more  difficult.  Dramatic  and  rapid  changes  in  the  structure  and  organization  of 
the  Army  present  significant  challenges  to  the  S3  to  handle  force  development  and 
modernization.  Doctrine  has  not  yet  been  developed  to  support  decision-making  regarding 
balancing  personnel  and  resource  allocation  with  mission  requirements.  In  addition,  the  scale  of 
the  changes  in  unit  structure  (i.e.,  installation-wide  implications)  makes  oversight  of  force 
accounting  difficult  because  a  great  deal  of  change  is  occurring  at  one  time.  Finally,  the  new  BN 
S3  is  relatively  inexperienced,  having  not  learned  much  about  the  day-to-day  demands  of  his  job 
from  previous  duty  positions  or  in  the  schoolhouse.  The  S3,  however,  usually  outranks  the  other 
officers  of  the  staff,  except  the  XO,  and  is  expected  to  bring  leadership  to  his  job  duties. 

Unit  Effectiveness  -  A  Proposed  Framework  for  Assessment 

To  assess  the  impact  that  S3-XO.net  could  potentially  have  on  unit  effectiveness,  a  red- 
amber-green  rating  system  analogous  to  the  one  used  earlier  in  this  report  was  used.  The  red- 
amber-green  criteria  for  each  aspect  of  unit  effectiveness  are  presented  in  the  table  below.  It  is 
important  to  note  that  the  assessment  criteria  presented  below  are  recommended  as  a  starting 
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point  and  that  several  of  the  assessment  criteria  presented  here  must  change  as  current  issues 
evolve.  In  addition,  these  criteria  may  not  be  exhaustive  or  reflect  the  areas  of  greatest  joint 
interest  to  the  Army  and  S3-XO.net  members.  Finally,  it  should  be  noted  that  the  proposed 
framework  is  for  evaluating  potential  impact  of  the  forum,  rather  than  assessing  impact  directly. 
The  assumption  behind  this  approach  is  that  (a)  it  is  more  feasible  and  more  tractable  to  assess 
the  potential  for  impact  rather  than  impact  directly;  and  (b)  if  direct  impact  is  of  particular 
interest,  the  proposed  approach  is  useful  for  determining  whether  taking  on  such  an  effort  would 
provide  useful  additional  information. 

Unit  Effectiveness  —  Assessment  of  S3-XO.net 

The  impact  of  S3-XO.net  on  BN/BDE  staff  effectiveness  was  not  formally  assessed  in  the 
present  investigation  in  part  because  status  ratings  of  conversation,  content,  and  connections 
were  all  red-amber.  Given  these  status  ratings,  it  was  unlikely  that  S3-XO.net  could  have  a 
potential  impact  status  greater  than  red  or  red-amber.  In  addition,  in  the  previous  section,  it  was 
determined  that  S3-XO.net  likely  has  not  yet  achieved  significant  individual-level  impact. 
Individual-level  impact  is  necessary  for  unit-level  impact  to  be  achieved.  Informal  assessment  of 
S3-XO.net  indicated  that  some  of  the  topics  identified  as  relevant  to  having  an  impact  on  unit 
effectiveness  were  present  in  the  forum,  including,  for  example,  S3  battle  rhythm,  TOC 
operations,  MDMP  training,  and  so  on.  However,  conversation  on  these  topics  tended  to  be  quite 
sparse. 


Formal  analysis  of  impact  can  be  expected  to  provide  information  above  and  beyond 
analysis  of  social  and  intellectual  capital  when  status  ratings  of  conversation,  content,  and 
connections  are  amber-green  or  green  and  substantial  impact  on  the  individual-level  has  been 
determined.  Impact  analysis  at  that  point  addresses  whether  social  and  intellectual  capital 
actively  developed  in  the  forum  can  address  recognized  shortfalls  in  unit  effectiveness  and 
whether  improvements  in  forum  member  competence  and  professionalism  generate 
improvements  in  unit  effectiveness.  Direct  assessment  of  impact  addresses  whether  relevant 
social  and  intellectual  capital  development  and  the  enhancement  of  individual  competenee  can 
overcome  the  cultural,  logistical,  administrative,  or  other  challenges  to  effective  unit 
performance. 


Unit  Effectiveness  -  Recommendations 

Our  recommendations  for  generating  discussion  and  knowledge  sharing  that  will  have  a 
greater  impact  on  unit  effectiveness  are  the  same  as  the  recommendations  for  enhancing  impact 
on  member  competence  and  professionalism.  If  the  facilitators  are  not  themselves  current 
members  of  the  target  membership,  then  they  must  otherwise  become  intimately  familiar  with 
the  needs  and  interests  of  the  members  and  organization  they  are  serving. 

S3-XO.net  Impact  on  Army  Effectiveness 

Managing  the  scope  of  organization-level  impact  assessment  requires  targeting  those 
aspects  of  organizational  effectiveness  that  present  common  or  current  challenges  to  the  target 
members  of  the  APF  to  be  assessed.  As  a  counterexample,  one  issue  the  Army  currently  faces  is 
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Table  23.  A  Proposed  Framework  for  Assessing  Potential  Impact  on  Unit  Effectiveness 
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meeting  recruiting  quotas.  Although,  in  theory,  participation  in  an  APF  may  address  recruiting 
challenges  (e.g.,  recruiters  could  use  such  an  APF  to  exchange  techniques  for  “getting  to  yes”  or 
for  locating  candidates  likely  to  enlist),  participation  in  an  APF  geared  towards  BN/BDE 
S3s/XOs  likely  will  not.  Because  the  target  membership  of  S3-XO.net  is  comprised  of  current 
and  future  S3s  and  XOs  of  BN/BDE  staffs,  improvements  in  organization-level  effectiveness 
should  therefore  address  organizational  issues  that  relate  to  management  and  performance  at  the 
BN/BDE  echelons. 

Feasibility  issues  make  using  the  quasi-experimental  approach  undesirable  for  assessing 
organization-level  impact  unless  there  is  definitive  evidence  suggesting  that  such  an  assessment 
would  provide  additional  information  beyond  a  more  feasible  approach.  It  is  sensible  to  use  a 
review  of  the  content  and  discussion  in  the  forum  as  a  proxy  for  direct  impact  assessment  before 
moving  on  to  more  complicated  and  costly  assessment.  The  following  section  presents  a  brief 
overview  of  selected  Army-wide  initiatives  and  their  implications  for  the  jobs  of  the  BN/BDE  S3 
and  XO.  We  highlight  areas  of  particular  challenge  using  our  findings  from  a  combination  of 
literature  review  and  interviews  with  subject  matter  experts  (see  Appendix  A  for  a  list  of  the 
subject  matter  experts  we  interviewed).  If  S3-XO.net  is  to  have  an  impact  on  organizational 
effectiveness  these  areas  should  represent  a  significant  proportion  of  the  productive  discussion 
and  knowledge  sharing  occurring  in  the  forum. 

The  Battalion,  The  Brigade,  and  Organizational  Initiatives 

Identifying  the  organization-wide  impact  of  activity  in  a  single  APF  is  exceedingly 
difficult  for  the  reasons  described  at  the  beginning  of  this  report.  It  should  be  possible,  however, 
to  articulate  what  affect  on  organizational  excellence  that  more  widespread  knowledge 
management  (KM)  practices  can  be  expected  to  have.  It  is  one  thing  to  say  that  KM  will  make 
the  Army  more  effective  and  another  to  articulate  how  this  is  to  occur  and  what  “effective” 
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means.  An  ongoing  analysis  of  BCKS  ,  conducted  by  researchers  at  the  Institute  for  Defense 
Analysis,  has  listed  four  ways  in  which  the  Army  KM  initiative  is  expected  to  influence 
organizational  excellence.  These  are  the  operational  goals  of  BCKS.  Three  of  these  goals  will  be 
discussed  briefly  below  because  of  their  applicability  to  APFs  in  particular. 

The  first  means  by  which  KM  is  expected  to  improve  organizational  effectiveness  is 
through  enhanced  learning.  Through  KM,  leader  development  will  be  enhanced  by  increased 
access  to  learning  content,  greater  opportunity  to  repeat  the  practice  of  critical  skills  and 
competencies,  and  more  opportunities  to  receive  performance  feedback.  Second,  KM  is  expected 
to  result  in  increased  leader  ability  to  make  intuitive  decisions.  Increased  opportunity  to  praetice 
skills  and  eompetencies  with  expert  feedback  using  a  wider  variety  of  environmental  conditions 
is  expected  to  help  leaders  improve  their  pattern  recognition  and  responses  to  eomplex  situations. 
Third,  KM  is  expected  to  result  in  the  development  of  more  adaptive  leaders.  Leaders’  ability  to 
reeognize  novel  situations  and  respond  effectively  will  be  increased  through  access  to  knowledge 
and  discussion  that  will  aid  in  understanding  the  current  situation  and  its  implications. 

The  above-described  operational  goals  for  BCKS  represent  high-level  goals  for  S3- 
XO.net.  The  key  challenges  faced  by  S3s/XOs  discussed  previously  may  be  addressed  by 

Recall  that  S3-XO.net  is  a  part  of  Leader  Network,  a  component  of  BCKS. 
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meeting  the  goals  of  enhanced  learning  and  more  adaptive  decision-making,  among  the 
development  of  other  skills,  competencies,  and  attitudes.  Further  articulation  of  the  expected 
organizational  impact  of  S3-XO.net  requires  identifying  the  organizational  initiatives  that 
enhanced  S3/XO  functioning  may  facilitate.  The  tremendous  amount  of  flux  in  the  Army  as  it 
transforms  to  the  Future  Force  has  direct  implications  for  the  leadership  and  management  of 
BDEs  and  BNs  because  these  units  are  significantly  affected  by  the  transformation  effort.  Below, 
two  key  aspects  of  Army  transformation  and  their  implications  for  the  S3/XO  are  briefly 
described.  The  general  means  by  which  enhanced  S3/XO  functioning  may  facilitate 
transformation  are  also  presented. 

Modularity 

Modularity  is  the  process  of  transforming  current  Army  structure  into  a  modular  force. 
The  overarching  goal  of  modularity  is  to  allow  combatant  commanders  more  flexible  use  of  land 
power  to  address  a  full  spectrum  of  operations  through  the  adaptive  selection  of  brigade 
“modules”  or  brigade  combat  teams  (BCTs)  to  meet  mission  requirements  (Smith,  2005).  Simply 
put,  modularity  is  changing  the  structure  of  the  Army  brigade  such  that  it  has  organic  combined 
arms  capability,  to  include  maneuver,  intelligence,  logistics,  and  signal,  among  other  functions. 
BCTs,  for  example  the  Stryker  brigade  combat  team  (SBCT),  will  be  standardized  in  their 
structure  and  in  their  SOPs  such  that  the  units  are  interchangeable  and  rapidly  deployable. 
Additional  multi-functional  support  brigade  “modules”  (e.g.,  aviation,  fires,  and  sustainment), 
with  standardized  component  “packages,”  will  also  be  available  to  the  combatant  commander  in 
order  to  task  organize  BCTs  to  meet  mission  requirements. 

Modularity  reduces  the  need  for  combatant  commanders  to  task  organize  brigades  with 
combined  arms  assets.  It  also  introduces  new  types  of  units,  such  as  the  maneuver  enhancement 
brigade  and  the  brigade  troops  battalion,  which  integrate  forces  that  were  once  separate.  In  the 
modular  force,  two  types  of  Unit  of  Employment  (UE),  UEx  and  UEy,  will  replace  the  division 
and  corps  echelons.  The  UE  echelons  will  not  have  a  set  structure,  but  will  be  assembled  flexibly 
on  the  basis  of  mission  requirements  and  unit  readiness. 

The  major  implications  of  modularity  for  the  S3  and  XO  include  significant  changes  in 
personnel  procedures,  equipment,  and  employment  tactics  and  associated  leader  and  staff 
training  requirements  (Delaney,  2004).  Moreover,  because  of  the  additional  organic  assets  in  the 
BCT,  the  structure  and  duties  of  the  BCT  staff  (which  is  analogous  to  a  division  staff;  Glenister, 
2002)  have  increased  in  scope  at  the  same  time  that  tactics  are  also  changing  due  to  new  types  of 
units  or  modified  unit  structures.  Modularity  represents  the  greatest  shift  in  force  structure  since 
the  1960’s  (Smith,  2005),  and  is  in  some  form  of  progress  at  installations  nationwide.  Some  more 
specific  implications  of  changes  to  the  current  force  for  the  job  of  the  S3  and  XO  are  described 
below.  The  discussion  below  is  not  complete  because  such  a  discussion  is  beyond  the  scope  of 
this  short-term  project,  but  it  is  meant  to  illustrate  the  nature  of  challenges  that  transformation 
presents  to  the  S3/XO  and  how  improving  S3/XO  capability  facilitates  this  Army-wide  initiative. 

ISR  in  the  BCT.  Two  types  of  BCT,  the  SBCT  and  the  infantry  BCT,  feature  a 
reconnaissance,  surveillance,  and  target  acquisition  (RSTA)  squadron  (Smith,  2005).  This 
organic  ISR  capability  in  one  sense  eases  ISR  planning  by  the  BCT  staff  because  such  planning 
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is  now  handled  by  the  RSTA  squadron  commander  and  staff  (Kasales  &  Gray,  2003).  However, 
close  coordination  must  occur  between  the  BCT  S3  and  RSTA  squadron  staff  such  that  the  ISR 
plan  is  maximally  integrated  with  the  brigade  maneuver  effort.  Moreover,  the  RSTA  squadron  is 
not  the  only  contributor  to  the  BCT  ISR  effort,  but  is  complemented  by  the  BCT  intelligence 
officer,  a  military  intelligence  (MI)  company,  and  infantry  battalion  MI  assets  (Toomey,  2003). 
The  BCT  S3  must  know  how  to  coordinate  these  resources  such  that  the  ISR  plan  supports  the 
BCT  maneuver  effort  and  that  the  information  collected  is  timely  and  relevant  (Kasales  &  Gray, 
2003)  -  a  task  that  was  already  difficult  with  reduced  complexity.  In  the  absence  of  well-defined 
tactics  for  employment,  the  RSTA  squadron  S3  must  fully  understand  the  capabilities  of  the 
range  of  assets  available  to  the  squadron  commander  (i.e.,  human  intelligence,  tactical  unmanned 
aerial  vehicles,  ground  surveillance  radar,  reconnaissance  squadrons,  radio  intercept.  Fox 
nuclear,  biological,  and  chemical  vehicles)  and  how  they  are  best  integrated  to  meet  the  squadron 
and  BCT  mission  requirements.  Integration  of  staff  efforts  across  echelons  is  an  additional 
training  challenge  for  both  the  BCT  and  RSTA  squadron  XO. 

Fires,  Targeting,  and  Effects.  Asymmetric  warfare  in  an  urban  environment  has  two 
broad  implications  for  the  use  of  fires  and  targeting.  First,  the  built-up  nature  of  the  urban 
environment  places  limitations  on  what  fires  assets  can  be  used  such  that  unnecessary  casualties 
and  destruction  to  infrastructure  are  minimized.  Second,  the  “battlefield”  on  which  the 
asymmetric  enemy  conducts  warfare  is  socio-political,  where  the  trust  and  cooperation  of  the 
local  populace  (and  other  key  stakeholders)  serves  as  “key  terrain;”  decisive  victory  requires  not 
annihilation  of  the  enemy  but  obviation  of  the  enemy  through  the  effective  targeting  and  denial 
of  this  key  terrain  to  the  enemy  (Cianciolo  &  DeWillie,  2004), 

The  Army  has  responded  to  the  need  for  effects-based  targeting  in  part  by  designing  the 
fires  and  effects  coordination  cell  (FECC),  which  is  responsible  for  coordinating  efforts  to 
achieve  lethal  and  non-lethal  effects  on  the  enemy  (Glenister,  2002).  The  FECC  is  situated 
within  the  headquarters  of  the  BCT,  serving  as  the  engine  that  drives  lethal  and  non-lethal 
targeting  for  the  unit,  planning  and  executing  fires  and  effects  tasks  in  collaboration  with  the 
other  BOSs  (FMI  3-09.42;  DOA,  2005).  It  is  an  organic,  standing  organization  with  information 
operations  capabilities,  which  differentiates  it  from  the  traditional  brigade  fire  support  element 
(FMI  3-09.42;  Glenister,  2002).  The  field  artillery  battalion  organic  to  the  BCT  supplies  the 
effects  coordinator  who  directs  the  FECC.  The  information  operations  section  of  the  FECC  is 
comprised  of  personnel  from  information  operations,  civil  affairs,  electronic  warfare,  and 
psychological  operations.  The  FECC  provides  direction  to  the  field  artillery  battalion’s  efforts  to 
support  the  BCT. 

The  introduction  of  the  FECC  requires  that  field  artillery  battalion  commanders  and  staffs 
address  the  task  of  fires  and  effects  task  coordination  (Glenister,  2002).  The  field  artillery 
battalion  S3  must  coordinate  closely  with  the  BCT  FECC  and  the  field  artillery  brigade 
headquarters  such  that  the  battalion  does  not  become  overtaxed  by  close  support,  shaping,  and 
counterstrike  responsibilities  (FMI  3-09.42).  This  S3  must  also  ensure  that  his  unit  is  trained  to 
effectively  execute  coordinated  fires  and  effects  tasks  as  the  FECC  and  battalion  commander 
direct.  The  BCT  S3  must  have  a  solid  understanding  of  how  coordinated  fires  and  effects  tasks 
will  support  the  maneuver  effort.  The  persistent  challenge  to  S3s  to  coordinate  maneuver  with 
fire  support  becomes  more  difficult  as  “overlapping  fires”  and  “clearing  effects”  take  on  new 
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meanings  when  attempting  to  achieve  social  or  political  effects  on  the  civilian  populace 
(Hamilton  &  Gist,  2004).  The  increased  complexity  of  the  staff  amplifies  the  challenge  to  the 
BCT  XO  to  train  and  synchronize  the  staff  effectively.  S3s  and  XOs  are  not  fully  supported  by 
doctrine  in  their  effort  to  adapt  to  and  exploit  the  FECC.  Interim  doctrine  exists  [FMI  (Field 
Manual  Interim)  3-09.42;  DOA,  2005],  but  it  is  not  fully  consistent  with  or  applicable  to  the 
tactics,  techniques,  and  procedures  adopted  in  practice  and  described  in  the  professional 
literature  (Glenister,  2002;  Hamilton  &  Gist,  2004).  In  addition,  the  interim  doctrine  appears  to 
confound  achieving  effects  with  performing  tasks,  a  confound  that  is  counterproductive  and 
widely  adopted. 

New  Units  -  The  Brigade  Troops  Battalion.  The  brigade  troops  battalion  (BTB)  is 
organic  to  certain  types  of  BCT  (e.g.,  the  heavy  BCT).  Its  C2  cell  (commander  and  staff) 
provides  a  centralized  means  for  coordinating  and  supporting  what  were  once  separate 
companies  and  detachments  typically  attached  to  a  traditional  brigade  (FMI  3-90.61;  DOA, 
2005).  The  BTB  is  comprised  of  four  organic  units  [BTB  headquarters  and  headquarters 
company  (HHC),  BCT  HHC,  a  military  intelligence  company,  and  a  signal  network  support 
company],  and  may  have  one  or  more  attached  elements,  including  civil  affairs  teams, 
psychological  operations,  public  affairs,  and  specialized  engineer  teams,  among  others  (FMI  3- 
90.61).  Its  general  mission  is  to  support  the  BCT  staff  during  planning,  to  carry  out  and  support 
BCT-designated  tasks,  to  support  the  BCT’s  command  posts  and  subordinate  units,  and  to 
provide  rear  area  and  base  security  (FM  3-90.61). 

Over  the  next  two  to  three  years,  the  commanders,  XOs,  and  S3s  of  the  BTB  will  come 
from  engineering  battalions  that  have  been  disbanded.  The  challenge  to  the  BTB  XO  to  train  and 
coordinate  a  staff  representing  multiple  functional  areas  is  not  a  new  one,  though  effective 
coordination  of  the  particular  specialties  present  in  the  BTB  requires  learning  of  unfamiliar 
functional  areas.  The  S3  must  also  learn  about  these  unfamiliar  functional  areas  in  order  to 
effectively  (a)  identify  the  training  needs  of  each  type  of  unit  and  of  the  BTB  overall;  (b)  provide 
this  training;  and  (c)  plan  and  support  the  coordinated  efforts  of  these  units.  That  the  combat 
engineering  battalion  is  being  largely  removed  from  the  force  structure  is  a  sore  point  for  several 
Army  engineers.  Some  engineering  S3s  and  XOs  assuming  a  duty  position  in  a  BTB  may  also 
have  to  strive  to  succeed  despite  low  morale  or  fhistration  with  changes  beyond  their  control. 

Digitization 

Digitization  is  the  process  of  enabling  battlefield  visualization,  collaborative  mission 
planning,  and  information  sharing  via  networked  computer  systems  (Barnett,  Meliza,  & 
McCluskey,  2001).  Digitized  tactical  operations  centers  feature  digital  situation  awareness 
displays  in  place  of  maps  and  overlays.  The  digitized  staff  plans  missions  from  distributed 
locations  on  the  battlefield  in  parallel  with  higher  and  lower  echelon  staffs.  They  create  and 
disseminate  orders  electronically.  As  is  the  case  with  all  technological  developments,  digitization 
addresses  some  past  challenges  and  also  creates  some  new  challenges,  including  new  equipment 
that  must  be  understood,  staffed,  and  maintained,  new  skills  that  must  be  learned,  new 
coordination  that  must  be  established,  new  doctrine  and  tactics  that  must  be  developed  and 
learned,  and  so  on  (Barnett  et  al.,  2001).  All  of  these  challenges  have  implications  for  the  S3  and 
XO.  Some  of  these  implications  are  summarized  below. 
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Maintaining  Connectivity.  Maintaining  connectivity  throughout  an  engagement  is  a 
difficult  task  made  more  difficult  by  digitization.  The  BN  and  BDE  staff  must  plan  the  location 
of  retransmission  vehicles  very  carefully  such  that  they  are  positioned  to  support 
communications  among  and  between  echelons  that  use  radio  systems  with  different  planning 
ranges  and  with  different  roles  during  the  engagement  (Cotter,  2002).  Digital  transmission  via 
Near-Term  Digital  Radio  is  required  to  keep  digital  BN  and  BDE  TOCs  from  reverting  back  to 
analog.  However,  Frequency  Modulation  and  Enhanced  Position  Location  Reporting  System 
radios  are  required  for  companies  to  communicate  with  each  other  and  with  their  parent  BNs. 
The  need  to  support  collaborative  planning,  battle  tracking,  and  information  dissemination  at  the 
BN  and  BDE  level  may  conflict  with  the  need  for  companies  to  communicate  with  their  parent 
BNs  using  FM  radio  during  the  close  fight,  especially  when  retransmission  resources  are  limited 
or  reduced  by  enemy  action  (Cotter,  2002).  The  BN  and  BDE  XO  must  ensure  that  their  signal 
officer  is  integrated  into  the  planning  process  such  that  retransmission  needs  are  resourced  and 
retransmission  vehicles  are  effectively  positioned  in  time  and  space  during  an  engagement  while 
also  secured  from  enemy  interference  or  attack. 

Digital  Skills  Training  and  Maintenance.  In  addition  to  maintaining  connectivity  and 
performing  basic  manipulations  of  the  interface,  users  of  digital  C2  equipment  must  exploit  the 
capabilities  of  the  technology  such  that  using  it  is  not  tantamount  to  doing  the  same  thing  with  a 
different  method.  The  use  of  digital  C2  equipment  should  result  in  fundamentally  doing  things 
differently.  It  requires  expertise  to  diagnose  and  correct  the  instability  of  the  tactical  Internet, 
expertise  that  is  not  resident  in  most  Soldiers.  For  these  reasons,  it  is  critical  that  the  S3  and  XO 
identify  and  train  individual  and  collective  digital  skills,  but  they  must  do  so  in  the  relative 
absence  of  doctrinal  or  other  support  for  performance  assessment  and  feedback  (though  see 
Leibrecht  et  al.,  2004).  Moreover,  digital  skills  are  highly  perishable,  so  S3s/XOs  must  be 
creative  in  determining  means  by  which  these  skills  can  be  practiced,  such  as  through  integrated 
use  of  the  Army  Battle  Command  System  (ABCS)  to  do  garrison  tasks  (e.g.,  to  monitor  garrison 
activities  such  as  training  (Brown  &  Dedrich,  2003;  Kasales  &  Gray,  2003).  The  S3/XO  must 
also  establish  digital  SOPs  to  address  information  management,  choices  about  what  information 
is  sent  when  due  to  bandwidth  constraints  and  limited  satellite  time  and/or  digital  gateways  for 
satellite  information  (Kasales  &  Gray,  2003;  Toomey,  2003). 

Information  Overload.  The  significant  increase  in  information  made  available  to  the  staff 
can  either  help  or  hinder  their  battle  tracking  and  support  of  the  commander’s  decision-making, 
depending  on  how  information  is  processed  and  managed  in  the  tactical  operations  center.  Too 
much  information  is  as  much  or  more  of  a  problem  as  too  little.  For  example,  the  ability  to  cut 
and  paste  information,  including  pictures  and  graphics,  from  digital  displays  enables  staffs  to 
produce  longer  and  longer  orders,  briefs,  and  other  products  which  are  not  necessarily  more 
infonnative  (Langley,  2004).  Moreover,  larger  files  consume  more  bandwidth,  thus  slowing 
down  not  only  information  processing  on  the  part  of  the  recipient  but  also  the  transmission  of  the 
file  itself  (Langley,  2004).  The  analysts  with  the  skills  required  to  process  the  large  volume  of 
information  produced  by  digital  systems  are  often  not  available  at  the  battalion  level  (Langley, 
2004),  so  the  BN  XO  must  ensure  that  some  other  means  is  in  place  to  manage  information 
coming  into  and  leaving  the  TOC.  The  previously  described  challenges  to  the  XO  and  S3  to 
manage  information  effectively  apply  to  handling  information  overload  in  the  digital  TOC. 
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Asynchronous  Digitization.  For  obvious  reasons,  digitization  cannot  occur  throughout  the 
Army  all  at  one  time.  The  result  is  that  some  units  are  digitized  ahead  of  others.  In  addition,  the 
command  and  control  systems  of  joint  forces  likely  will  not  have  compatible  computer  networks. 
The  implication  is  that  units  fighting  together  may  not  be  able  to  capitalize  on  digitization 
because  digital  systems  are  not  available  to  everyone.  The  responsibility  rests  with  the  XO  and 
S3  to  ensure  that  units  with  differing  digital  capability  can  still  be  synchronized  and  employed 
according  to  the  commander’s  intent. 

Army  Effectiveness  -  A  Proposed  Framework  for  Assessment 

As  with  the  previously  proposed  frameworks  for  assessment,  a  red-amber-green  rating 
system  was  used  as  the  format  for  the  present  proposed  framework,  and  it  is  illustrated  in  the 
table  below.  The  present  criteria  used  to  assess  potential  impact  on  Army  effectiveness  must 
evolve  to  reflect  current  issues.  In  addition,  these  criteria  are  by  no  means  exhaustive  and  may 
not  reflect  the  areas  of  greatest  joint  interest  to  the  Army  and  S3-XO.net  members.  A  more 
thorough  assessment  of  the  link  between  key  organizational  objectives  and  the  jobs  of  the 
BN/BDE  S3/XO  was  outside  the  scope  of  this  research  and  is  recommended  for  future 
assessment  of  S3-XO.net. 

Army  Effectiveness  -  Assessment  o/S3-XO.net 

The  impact  of  S3-XO.net  on  Army  effectiveness  was  not  formally  assessed  in  the  present 
research  for  three  reasons.  First,  status  ratings  of  conversation,  content,  and  connections  were  all 
red-amber.  Given  these  status  ratings,  it  is  unlikely  that  S3-XO.net  could  have  a  potential  impact 
status  greater  than  red  or  red-amber.  Second,  in  the  preceding  two  sections,  it  was  determined 
that  S3-XO.net  likely  has  not  yet  achieved  significant  individual-  or  unit-level  impact.  Third, 
even  if  social  and  intellectual  capital  were  actively  developed  in  S3-XO.net,  the  APF  has  been 
active  for  less  than  three  years.  This  time  period  is  likely  too  short  to  expect  a  discemable  impact 
at  the  organizational  level,  especially  considering  the  relatively  small  percentage  (3%)  of  the 
intended  target  audience  of  the  forum  currently  holding  membership. 

Informal  assessment  of  S3-XO.net  indicated  that  some  of  the  topics  identified  as  relevant 
to  having  an  impact  on  Army  effectiveness  are  present  in  the  forum,  including,  for  example, 
command  post  of  the  future,  fires  and  effects  coordination  cell,  combined  arms  training  in  the 
unit  of  action,  leadership  challenges  in  transformation,  and  so  on.  However,  conversation  about 
these  topics  tended  to  be  quite  sparse. 

Army  Effectiveness  -  Recommendations 

Our  recommendations  for  generating  discussion  and  knowledge  sharing  that  will  have  a 
greater  impact  on  Army  effectiveness  are  the  same  as  the  recommendations  for  enhancing  impact 
on  member  competence  and  professionalism  and  unit  effectiveness.  Facilitators  must  be 
intimately  familiar  with  the  needs  and  interests  of  the  members  and  organization  they  are 
serving. 
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APPENDIX  C  -  CONDUCTING  ASSESSMENT  OF  THE  ARMY  PROFESSIONAL  FORUM 


Regular,  formal  assessment  is  the  means  by  whieh  an  organization  ensures  the  health  and 
impaet  of  its  knowledge-management  initiatives  (Hanley,  2003;  Crager  &  Lemons,  2003;  Lesser 
&  Storck,  2001 ;  Department  of  the  U.S.  Navy,  2001).  Assessment  enables  decision  makers  to 
effectively  allocate  limited  resources  to  the  initiative  by  (a)  identifying  the  components  of  the 
initiative  that  require  additional  investment;  and  (b)  providing  insight  into  how  investment  could 
strengthen  vulnerable  aspects  of  the  initiative.  In  addition,  assessment  reveals  the  degree  to 
which  investment  in  the  initiative  can  be  expected  to  payoff  in  terms  of  organizational 
performance. 

Organizational  decision  makers  must  resist,  however,  confusing  the  means  for  capturing 
impact  with  impact  itself,  else  the  knowledge-sharing  initiative  will  become  incapable  of 
adapting  to  changes  in  the  organizational  environment.  As  an  analogy,  a  test  of  academic 
achievement  may  be  used  to  capture  the  impact  of  an  educational  program,  but  test  scores  should 
never  be  considered  the  outcome  of  the  educational  program.  When  this  happens  (and  it  often 
does),  focus  shifts  away  from  meeting  environmental  demands  for  intellectual  development 
toward  meeting  an  artificial  standard  for  performance.  Distancing  from  the  environment 
decreases  awareness  and  adaptability.  Metrics  for  assessing  the  impact  of  a  knowledge  sharing- 
initiative  must  be  fiexible  and  must  always  reflect  the  environmental  demands  that  pressure  the 
initiative  to  evolve.  The  metrics  and  techniques  presented  in  this  report  and  appendix  represent  a 
starting  point  for  thoughtful  analysis  of  Army  Professional  Forums  (APFs),  an  example  of  how  it 
can  be  accomplished  in  response  to  both  current  environmental  demands  and  in  accordance  with 
lasting  and  generalizable  best  practice  in  assessment. 

The  level  of  understanding  about  the  health  and  impact  of  a  discussion  forum  that  can  be 
accomplished  in  a  given  time  is  determined  by  the  focus  and  efficiency  of  the  assessment 
process.  Focus  is  enhanced  by  a  thorough  understanding  of  the  forum’s  online  meeting  space,  the 
needs  of  the  population  the  forum  is  intended  to  serve,  the  needs  of  the  organization,  and  how  the 
forum  is  expected  to  have  an  impact.  For  this  reason,  individuals  conducting  assessment  should 
be  educated  in  knowledge-sharing  theory  and  practice  and  in  the  Army’s  functioning  and 
organizational  structure,  particularly  as  it  relates  to  the  intended  membership  of  the  forum. 
Although  independent,  assessors  should  work  closely  with  the  forum  leadership,  facilitators,  and 
active  members  of  the  forum  to  ensure  that  assessment  directs  forum  cultivation  rather  than 
imposing  top-down  requirements.  Efficiency  is  enhanced  by  standardization  and,  where  possible, 
automated  data  collection.  Several  of  the  metrics  presented  in  this  report  are  designed  to  be 
generalizable,  and  provide  standardized  criteria  for  assessing  APFs.  In  the  table  below,  we 
present  a  broad  overview  of  conducting  assessment  of  APFs  using  our  assessment  framework. 

We  address  how  it  is  done,  how  often  it  needs  to  be  done,  how  long  it  takes,  and  whether  or  not  it 
can  be  automated. 
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How  is  it  done? 


Can  it  be 
automated? 


How  often  does  it  How  long 
need  to  be  done?  does  it  take? 


S;' ^  T'-'  '"C -'vf-r-rPLAN- 

Review  of  plan  documentation 

As  events  warrant 

0.5  hr 

No 

WssmsmmM 

Review  of  plan  documentation 

As  events  warrant 

0.5  hr 

No 

Description  of 
Target  Audience 

Review  of  plan  documentation 

As  events  warrant 

0.5  hr 

No 

Statement  of 

Norms,  Values,  and 
Conventions 

Review  of  plan  documentation 

As  events  warrant 

0.5  hr 

No 

Specification  of 

Roles  and 
Responsibilities 

Review  of  plan  documentation 

As  events  warrant 

1  hr 

No 

Functional  and 

Technical 

Specification 

Review  of  plan  documentation 

As  events  warrant 

1  hr 

No 

Comments 

The  plan  should  be  assessed  before  the  forum  is  launched— preferably  by  multiple  assessors 
to  ensure  the  objectivity  and  reliability  of  the  rating— and  after  significant  changes  in  forum 
membership  are  made  (e.g.,  a  sub-community  breaks  off  to  form  its  own  forum)  or 
immediately  after  significant  changes  in  technical  architecture  are  made  (e.g.,  adding 
synchronous  collaboration  capability).  Time  estimates  assume  one  assessor.  Two  assessors 
would  approximately  double  the  time  estimate.  After  the  plan  is  independently  assessed,  the 
findings  should  be  discussed  with  the  forum  leadership  and  recommendations  for 
strengthening  the  plan  should  be  provided.  If  the  initial  plan  requires  significant 
modification,  assessment  should  be  conducted  immediately  after  the  plan  is  revised  and 
prior  to  forum  launch. 

..  PERSONNEL^:-.'- ■■■'■ 

Leadership 

Interviews  with  forum  leaders 

2  hr  +  N  hrs 

No 

Core  Group 

Interviews  with  core  group; 
Biographical  data  collection 

Every  6  months 

4  hr  ■+•  N  hrs 

No 

Facilitators 

Interviews  with  facilitators;  Analysis 
of  facilitator  activity  in  the  forum 

Every  6  months 

4  hr  +  N  hrs 

No 

Technical  Support 

Interviews  with  technical  support 
staff 

Every  6  months 

2  hr  +  N  hrs 

No 

Comments 

Initial  assessment  of  personnel  should  be  conducted  within  six  months  after  forum  launch. 

The  time  required  to  conduct  the  interviews  for  each  personnel  category  is  two  hours  plus 
one  hour  times  the  number  of  people  interviewed.  This  estimate  allows  one  hour  for  each 
interview  and  two  hours  to  collate  and  summarize  interview  findings.  Where  other  data  are 
collected,  two  additional  hours  are  estimated  to  conduct  data  collection.  For  quiet  forums, 
two  hours  will  allow  complete  analysis.  For  very  active  forums,  two  hours  will  allow 
analysis  of  a  representative  sampling  of  data.  Time  estimate  does  not  include  time  devoted 
to  scheduling  interviews  or  downloading  conversation  posts  into  a  main  database  used  for 
analysis.  This  could  be  done  automatically  with  databasing  and  scripting  techniques.  After 
personnel  are  independently  assessed,  the  findings  should  be  discussed  with  the  forum 
leadership  and  recommendations  for  enhancing  personnel  support  should  be  provided. 

FUNCTIONAL  REQUIREMENTS 

Locating  the 

Forum 

Query  forum  leaders  and/or 
facilitators;  External  search  for  forum 

As  events  warrant 

2  hr 

No 

Joining  the  Forum 

Query  facilitators;  Survey  members 

As  events  warrant 

1.5  hr 

Yes/No 

Tone  in  the  Forum 

Analysis  of  online  meeting  space 

Every  6  months 

1  hr 

No 

Conversation 

Support 

Analysis  of  online  meeting  space 

As  events  warrant 

1  hr 

No 
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Locating  Content 


As  events  warrant 


Yes/No 


Analysis  of  online  meeting  space; 

Survey  members  _ 


Analysis  of  online  meeting  space  |  As  events  warrant  |  1 .5  hr  |  No 


Initial  assessment  of  functional  requirements,  with  the  exception  of  surveying  members, 
should  occur  prior  to  forum  launch  as  a  sort  of  usability  test.  Follow-on  assessment  should 
occur  as  significant  changes  are  made  to  the  design  of  the  online  meeting  space  (e.g.,  new 
content  taxonomy)  or  to  how  it  is  accessed.  Surveying  members  can  be  automated  through 
pop-up  survey  questions  administered  in  the  online  meeting  space.  Time  estimates  presented 
above  are  likely  to  be  shorter  for  follow-on  assessments.  Time  estimates  for  surveying 
members  do  not  include  the  time  required  to  automate  survey  administration  or  to  modify 
survey  questions  from  those  presented  in  Appendix  C,  but  do  assume  time  required  for  data 
entry  and  summarization.  Time  estimates  apply  to  surveying  a  representative  sample  of 
forum  members.  After  functional  requirements  are  independently  assessed,  the  findings 
should  be  discussed  with  the  forum  leadership  and  facilitators,  and  recommendations  for 
enhancing  functional  requirements  should  be  provided.  If  initial  functional  requirements 
need  significant  modification,  assessment  should  be  conducted  immediately  after  they  are 
revised  and  prior  to  forum  launch.  _ 


Membership 

Composition 


Participating 

Members 


Identifying  Experts 


Facilitation  of 
Connections 


Connection  Quali 


Shared  Narratives 


Analysis  of  online  meeting  space  Every  6  months 


Analysis  of  online  meeting  space  Every  6  months 


Analysis  of  online  meeting  space; 
Survey  members 


Every  6  months 


Yes/No 


Query  facilitators;  Survey  members  Every  6  months  2  hr 


Survey  members  _ |  Every  6  months  | _ 3  hr _ |  Yes 


Initial  assessment  of  connections  should  occur  within  6  months  of  forum  launch.  All  time 
estimates  assume  that  assessment  is  not  automated.  Time  estimate  for  membership 
composition  includes  time  for  data  downloading  and  cleaning.  Surveying  members  can  be 
automated  through  pop-up  survey  questions  administered  in  the  online  meeting  space.  Time 
estimates  do  not  include  the  time  required  to  automate  survey  administration  or  to  modify 
survey  questions  from  those  presented  in  Appendix  C,  but  do  include  time  for  data  entry  and 
summarization.  Time  estimates  apply  to  surveying  a  representative  sample  of  forum 
members.  Data  collection  on  membership  composition,  participating  members  and 
identifying  experts  could  be  automated  through  databasing  and  scripting  techniques.  After 
coimections  are  independently  assessed,  the  findings  should  be  discussed  with  the  forum 
leadership  and  facilitators,  and  recommendations  for  enhancing  connections  should  be 
provided.  _ 


CONTEXT..;.;- ■ 


Analysis  of  online  meeting  space  |  Every  6  months  |  0.5  hr  |  No 


Survey  members 


Every  6  months 


Initial  assessment  of  connections  should  occur  within  6  months  of  forum  launch.  Time 
estimate  for  surveying  members  include  time  for  data  entry  and  summarization.  Surveying 
members  can  be  automated  through  pop-up  survey  questions  administered  in  the  online 
meeting  space.  Time  estimates  do  not  include  the  time  required  to  automate  survey 
administration  or  to  modify  survey  questions  from  those  presented  in  Appendix  C,  but  do 
include  time  for  data  entry  and  summarization.  Time  estimates  apply  to  surveying  a 
representative  sample  of  forum  members.  After  context  is  independently  assessed,  the 
findings  should  be  discussed  with  the  forum  leadership  and  facilitators,  and 
recommendations  for  enhancing  context  should  be  provided^ _  _ 
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Contributing 

Members 


Content 

Organization 


Content  Quality 


Professional  Links 


Analysis  of  online  meeting  space 


Analysis  of  online  meeting  space 


Analysis  of  online  meeting  space; 
Survey  members 


Analysis  of  online  meeting  space 


Analysis  of  online  meeting  space; 
Survey  members 


Analysis  of  online  meeting  space 


Every  6  months 


Every  6  months 


Every  6  months 


Every  6  months 


Every  6  months 


Yes 


No 


Yes 


Yes 


Yes/No 


No 


Indirect 

Assessment 


Every  6  months 


Initial  assessment  of  content  should  occur  within  6  months  of  forum  launch.  Time  estimate 
for  contributing  members  assumes  data  downloading  and  cleaning  for  membership 
composition  has  been  completed.  Surveying  members  can  be  automated  through  pop-up 
survey  questions  administered  in  the  online  meeting  space.  Time  estimates  do  not  include 
the  time  required  to  automate  survey  administration  or  to  modify  survey  questions  from 
those  presented  in  Appendix  C,  but  do  include  time  for  data  entry  and  summarization.  Time 
estimates  apply  to  surveying  a  representative  sample  of  forum  members.  Data  collection  on 
contributing  members  and  content  activity  could  be  automated  through  databasing  and 
scripting  techniques.  For  quiet  forums,  the  time  estimate  for  content  synthesis  and  content 
quality  will  allow  complete  analysis.  For  very  active  forums,  estimates  will  allow  analysis  of 
a  representative  sampling  of  data.  Time  estimate  does  not  include  the  time  to  modify  the 
rubric  presented  in  this  report  or  to  create  a  new  rubric.  Where  possible,  two  assessors 
should  be  used  to  assess  content  synthesis  and  content  quality  and  their  assessments 
compared  and  combined.  After  content  is  independently  assessed,  the  findings  should  be 
discussed  with  the  forum  leadership  and  facilitators,  and  recommendations  for  enhancing 
content  should  be  provided. 


.  CONVERSATION 


Participating 

Members 

— 

Analysis  of  online  meeting  space 

Every  6  months 

0.5  hr 

Yes 

Analysis  of  online  meeting  space 

Every  6  months 

5hr 

Yes/No 

Analysis  of  online  meeting  space; 
Survey  members 

Every  6  months 

8hr 

Yes/No 

Initial  assessment  of  conversation  should  occur  within  6  months  of  forum  launch.  Time 
estimate  for  contributing  members  assumes  data  downloading  and  cleaning  for  membership 
composition  has  been  completed.  Surveying  members  can  be  automated  through  pop-up 
survey  questions  administered  in  the  online  meeting  space  The  time  estimate  does  not 
include  the  time  required  to  automate  survey  administration  or  to  modify  survey  questions 
from  those  presented  in  Appendix  C,  but  do  include  time  for  data  entry  and  summarization. 
Time  estimates  apply  to  surveying  a  representative  sample  of  forum  members.  Data 
collection  on  participating  members  and  conversation  activity  could  be  automated  through 
databasing  and  scripting  techniques.  For  quiet  forums,  the  time  estimate  for  conversation 
quality  will  allow  complete  analysis.  For  very  active  forums,  estimates  will  allow  analysis  of 
a  representative  sampling  of  data.  Time  estimate  does  not  include  the  time  to  modify  the 
rubric  presented  in  this  report  or  to  create  a  new  rubric.  Where  possible,  two  assessors 
should  be  used  to  assess  conversation  quality  and  their  assessments  compared  and 
combined.  Time  estimates  do  not  include  time  to  download  conversation  posts  into  a  main 
database  used  for  analysis.  This  could  be  done  automatically  with  databasing  and  scripting 
techniques.  After  conversation  is  independently  assessed,  the  findings  should  be  discussed 
with  the  forum  leadership  and  facilitators,  and  recommendations  for  improving  conversation 
should  be  provided.  _ 


INDIVroUAL IMPACT,  , 


Analysis  of  online  meeting  space  Every  6  months 


Unknown 
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Direct  Assessment 


As  events  warrant 


Unknown 


Yes/No 


Indirect 

Assessment 


Direct  Assessment 


Special  purpose  measures;  Survey 
members 


The  time  estimated  to  conduct  indirect  assessment  is  unknown  because  it  is  unknown  how 
long  it  will  take  to  determine  what  content  should  be  present  in  the  forum  to  aid  in  the 
development  of  individual  competence  and  professionalism.  This  will  be  at  least  slightly 
different  for  each  forum.  Once  relevant  content  is  determined,  the  estimated  time  to  conduct 
assessment  will  depend  on  how  much  content  is  relevant  and  sampled.  A  forum  plan  with  a 
solid  statement  of  envisioned  impact  and  facilitators/assessors  familiar  with  the  current 
Army  will  facilitate  indirect  assessment.  The  time  estimated  to  conduct  direct  assessment  is 
unknown  because  it  is  unknown  how  long  it  will  take  to  develop  special  purpose  measures 
(e.g.,  tactical  games,  situational  judgment  scenarios)  and  implement  them  in  the  online 
meeting  space.  Surveying  members  can  be  automated  through  pop-up  survey  questions 
administered  in  the  online  meeting  space,  but  survey  questions  will  have  to  change  as 
content  or  emphasis  changes.  Direct  assessment  should  only  be  conducted  if  the  findings 
from  indirect  assessment  suggest  it  is  warranted.  After  individual  impact  is  independently 
assessed,  the  findings  should  be  discussed  with  the  forum  leadership  and  facilitators,  and 
recommendations  for  improving  impact  should  be  provided.  _ 


.  UNIT  IMPACIV 


Analysis  of  online  meeting  space  Every  6  months  Unknown 


Unit  performance  observations  As  events  warrant  Unknown  No 


The  time  estimated  to  conduct  indirect  assessment  is  unknown  because  it  is  unknown  how 
long  it  will  take  to  determine  what  content  is  necessary  to  aid  in  the  enhancement  of  unit 
effectiveness.  This  will  be  different  for  each  forum.  Once  relevant  content  is  determined,  the 
estimated  time  to  conduct  assessment  will  depend  on  how  much  content  is  relevant  and 
sampled.  A  forum  plan  with  a  solid  statement  of  envisioned  impact  and  facilitators/assessors 
familiar  with  the  current  Army  will  facilitate  indirect  assessment.  The  time  estimated  to 
conduct  direct  assessment  is  unknown  because  it  is  unknown  how  long  it  will  take  to  arrange 
and  conduct  observations.  Direct  assessment  should  only  be  conducted  if  the  findings  from 
indirect  assessment  suggest  it  is  warranted.  After  unit  impact  is  independently  assessed,  the 
findings  should  be  discussed  with  the  forum  leadership  and  facilitators,  and 
recommendations  for  improving  impact  should  be  provided. 


;  O  IMPACT  V  S; 


Indirect 

Assessment 

Analysis  of  online  meeting  space 

Every  6  months 

Unknown 

No 

Direct  Assessment 

Organizational  performance 
observations;  Archival  data  collection 

As  events  warrant 

Unknown 

No 

Comments 

The  time  estimated  to  conduct  indirect  assessment  is  unknown  because  it  is  unknown  how 
long  it  will  take  to  determine  what  content  is  necessary  to  aid  in  the  enhancement  of 
organizational  effectiveness.  This  will  be  different  for  each  forum.  Once  relevant  content  is 
determined,  the  estimated  time  to  conduct  assessment  will  depend  on  how  much  content  is 
relevant  and  sampled.  A  forum  plan  with  a  solid  statement  of  envisioned  impact  and 
facilitators/assessors  familiar  with  the  current  Army  will  facilitate  indirect  assessment.  The 
time  estimated  to  conduct  direct  assessment  is  unknown  because  it  is  unknown  how  long  it 
will  take  to  arrange  and  conduct  observations  or  collect  archival  data.  Direct  assessment 
should  only  be  conducted  if  the  findings  from  indirect  assessment  suggest  it  is  warranted. 
After  organizational  impact  is  independently  assessed,  the  findings  should  be  discussed  with 
the  forum  leadership  and  facilitators,  and  recommendations  for  improving  impact  should  be 
provided. 
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APPENDIX  D  -  FORUM  MEMBER  SURVEY 
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APPENDIX  E  -  ACRONYMS 


AKO 

APF 

ATLDP 

Army  Training  and  Leader  Development  Panel 

BCKS 

Battle  Command  Knowledge  System 

BCT 

brigade  combat  team 

BDE 

brigade 

BN 

battalion 

BOS 

battlefield  operating  system 

BTB 

brigade  troops  battalion 

C2 

command  and  control 

CAC 

Combined  Arms  Center 

CALL 

Center  for  Army  Lessons  Learned 

CCIR 

commander’s  critical  information  requirements 

CG 

commanding  general 

CGSC 

Command  and  General  Staff  College 

COA 

course  of  action 

Cop 

community  of  practice 

DOA 

U.S.  Department  of  the  Army 

DSM 

decision  support  matrix 

DST 

decision  support  template 

FECC 

fires  and  effects  coordination  cell 

FM 

field  manual 

FMI 

field  manual  interim 

FORSCOM 

U.S.  Army  Forces  Command 

ISR 

intelligence,  surveillance,  and  reconnaissance 

KM 

knowledge  management 

MDMP 

military  decision-making  process 

METL 

mission  essential  task  list 

MI 

military  intelligence 

MTDA 

modification  table  of  distribution  and  allowances 

MTOE 

modification  table  of  organization  and  equipment 

NCO 

noncommissioned  officer 

OIF  MEAT 

Operation  Iraqi  Freedom  Mental  Health  Advisory  Team 

RSTA 

reconnaissance,  surveillance,  and  target  acquisition 
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S3 

operations  officer 

SECT 

Stryker  brigade  combat  team 

SOP 

standing  operating  procedure 

TACSOP 

tactical  standing  operating  procedures 

TOC 

tactical  operations  center 

UE 

unit  of  employment 

USAIS 

U.S.  Army  Infantry  School 

XO 

executive  officer 
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